Jul 28 17:08:37.130173 ip-10-0-136-215 systemd[1]: Starting Kubernetes Kubelet... Jul 28 17:08:37.498877 ip-10-0-136-215 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:37.498877 ip-10-0-136-215 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 17:08:37.498877 ip-10-0-136-215 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:37.498877 ip-10-0-136-215 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 17:08:37.498877 ip-10-0-136-215 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:08:37.501096 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.501005 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 17:08:37.504544 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504527 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:37.504544 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504544 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504547 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504551 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504554 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504557 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504560 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504564 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504567 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504570 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504573 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504576 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504578 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504581 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504584 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504586 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504589 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504592 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504594 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504597 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504600 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:37.504606 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504602 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504605 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504608 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504611 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504614 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504618 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504621 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504623 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504626 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504628 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504631 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504633 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504636 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504639 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504642 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504644 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504647 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504649 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504652 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:37.505095 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504655 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504659 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504663 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504666 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504669 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504672 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504675 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504677 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504681 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504685 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504688 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504691 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504694 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504696 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504699 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504702 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504704 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504707 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504709 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:37.505573 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504712 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504715 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504717 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504720 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504723 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504725 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504728 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504730 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504733 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504735 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504738 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504743 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504746 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504749 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504752 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504755 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504758 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504761 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504763 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:37.506045 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504765 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504769 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504772 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504774 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504777 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504779 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504782 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504785 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504788 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504790 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504793 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504795 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504798 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504800 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504803 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504806 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504808 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504811 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504813 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:37.506516 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504816 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.504818 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505563 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505571 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505575 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505578 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505581 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505584 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505587 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505591 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505595 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505599 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505601 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505604 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505608 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505612 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505615 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505618 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505621 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:37.507006 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505624 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505626 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505629 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505632 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505634 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505637 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505640 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505642 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505645 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505647 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505650 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505653 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505655 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505658 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505661 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505663 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505666 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505668 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505671 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:37.507477 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505673 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505675 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505678 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505680 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505683 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505686 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505689 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505692 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505695 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505698 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505700 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505703 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505705 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505708 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505712 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505715 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505717 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505720 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505722 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:37.507967 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505725 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505727 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505729 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505732 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505734 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505737 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505740 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505742 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505745 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505747 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505750 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505753 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505756 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505758 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505761 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505763 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505766 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505768 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505771 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505774 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:37.508432 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505778 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505781 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505783 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505786 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505790 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505793 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505795 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505798 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505800 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505803 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505805 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505808 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505810 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505812 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505815 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505817 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505820 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505822 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505840 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:37.508930 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505843 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505846 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505848 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505851 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.505854 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506238 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506244 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506247 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506249 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506252 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506254 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506257 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506260 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506265 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506269 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506272 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506276 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506279 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506281 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506285 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:37.509392 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506288 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506291 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506294 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506297 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506300 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506303 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506306 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506309 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506312 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506315 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506317 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506320 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506322 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506325 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506327 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506330 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506333 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506336 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506338 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506340 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:37.509911 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506343 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506345 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506348 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506355 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506357 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506360 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506363 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506366 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506369 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506371 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506374 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506377 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506379 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506382 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506384 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506387 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506389 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506392 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506395 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506397 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:37.510405 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506400 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506403 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506407 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506410 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506412 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506415 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506418 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506421 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506423 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506426 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506428 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506431 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506433 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506436 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506438 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506442 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506444 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506447 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506449 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:37.510918 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506452 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506455 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506458 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506460 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506463 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506465 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506468 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506470 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506473 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506475 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506478 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506480 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506483 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506485 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506488 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506493 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506495 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506498 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506501 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:37.511413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506503 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506506 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506508 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506511 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506513 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.506517 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506601 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506611 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506620 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506628 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506635 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506640 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506645 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506649 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506658 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506661 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506665 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506668 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506671 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506674 2570 flags.go:64] FLAG: --cgroup-root="" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506677 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506680 2570 flags.go:64] FLAG: --client-ca-file="" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506683 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 28 17:08:37.511890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506686 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506691 2570 flags.go:64] FLAG: --cluster-domain="" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506694 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506697 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506700 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506705 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506709 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506712 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506715 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506719 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506722 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506725 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506728 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506731 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506734 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506739 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506742 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506745 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506749 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506752 2570 flags.go:64] FLAG: --enable-server="true" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506755 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506759 2570 flags.go:64] FLAG: --event-burst="100" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506762 2570 flags.go:64] FLAG: --event-qps="50" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506765 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 17:08:37.512430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506769 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506772 2570 flags.go:64] FLAG: --eviction-hard="" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506776 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506778 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506782 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506785 2570 flags.go:64] FLAG: --eviction-soft="" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506788 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506791 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506794 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506797 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506800 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506803 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506806 2570 flags.go:64] FLAG: --feature-gates="" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506811 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506814 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506817 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506821 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506835 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506838 2570 flags.go:64] FLAG: --help="false" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506841 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506845 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506848 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506851 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506854 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 17:08:37.513092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506858 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506861 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506865 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506868 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506871 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506874 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506877 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506880 2570 flags.go:64] FLAG: --kube-reserved="" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506883 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506887 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506890 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506893 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506896 2570 flags.go:64] FLAG: --lock-file="" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506899 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506901 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506904 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506910 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506913 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506915 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506918 2570 flags.go:64] FLAG: --logging-format="text" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506921 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506926 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506929 2570 flags.go:64] FLAG: --manifest-url="" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506932 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506936 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 17:08:37.513709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506939 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506943 2570 flags.go:64] FLAG: --max-pods="110" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506946 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506949 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506952 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506956 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506958 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506961 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506964 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506975 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506978 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506981 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506984 2570 flags.go:64] FLAG: --pod-cidr="" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506987 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506993 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506995 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.506999 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507002 2570 flags.go:64] FLAG: --port="10250" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507005 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507008 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06d8b61d87e019cba" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507012 2570 flags.go:64] FLAG: --qos-reserved="" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507015 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507018 2570 flags.go:64] FLAG: --register-node="true" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507021 2570 flags.go:64] FLAG: --register-with-taints="" Jul 28 17:08:37.514331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507025 2570 flags.go:64] FLAG: --registry-burst="10" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507028 2570 flags.go:64] FLAG: --registry-qps="5" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507031 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507033 2570 flags.go:64] FLAG: --reserved-memory="" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507038 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507041 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507044 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507047 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507050 2570 flags.go:64] FLAG: --runonce="false" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507053 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507057 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507060 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507062 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507065 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507068 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507072 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507075 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507078 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507081 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507084 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507087 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507091 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507094 2570 flags.go:64] FLAG: --system-cgroups="" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507097 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507102 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 17:08:37.514935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507105 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507108 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507112 2570 flags.go:64] FLAG: --tls-min-version="" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507115 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507118 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507121 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507124 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507127 2570 flags.go:64] FLAG: --v="2" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507131 2570 flags.go:64] FLAG: --version="false" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507135 2570 flags.go:64] FLAG: --vmodule="" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507139 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507143 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507276 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507279 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507282 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507285 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507287 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507290 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507293 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507295 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507298 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507300 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:37.515543 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507303 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507305 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507308 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507311 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507314 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507316 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507319 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507322 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507325 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507327 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507330 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507332 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507336 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507338 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507341 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507344 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507346 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507350 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507354 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507357 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:37.516093 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507359 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507363 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507366 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507369 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507372 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507374 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507377 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507380 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507382 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507386 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507390 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507393 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507396 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507399 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507401 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507404 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507407 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507409 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507412 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507414 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:37.516611 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507417 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507420 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507422 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507425 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507427 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507432 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507435 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507438 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507441 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507443 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507446 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507449 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507451 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507456 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507458 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507461 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507464 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507466 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507468 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:37.517155 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507471 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507473 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507476 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507478 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507481 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507484 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507487 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507490 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507493 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507496 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507498 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507501 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507504 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507507 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507510 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507512 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507515 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507518 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507520 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507530 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:37.517661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507533 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507536 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507538 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507541 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507543 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507545 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507549 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507552 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507554 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.507557 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.507566 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.514198 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.514214 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.514234 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 17:08:37.518176 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.514240 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514304 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514309 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514312 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514316 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514319 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514322 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514325 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514328 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514332 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514336 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514341 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514344 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514347 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514351 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514354 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514357 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514360 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:37.518529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514363 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514365 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514368 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514371 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514373 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514376 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514379 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514382 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514384 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514388 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514390 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514393 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514396 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514399 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514402 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514404 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514407 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514410 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514413 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514416 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514418 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:37.519148 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514421 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514423 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514427 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514429 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514432 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514434 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514437 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514441 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514445 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514448 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514450 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514453 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514455 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514458 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514461 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514463 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514465 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514469 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514471 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514474 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:37.519668 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514477 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514480 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514483 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514485 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514489 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514491 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514494 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514497 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514499 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514502 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514504 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514507 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514510 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514512 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514515 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514518 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514520 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514523 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514526 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514528 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:37.520413 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514530 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514533 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514536 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514538 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514541 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514543 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514546 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514549 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514551 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514554 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514556 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514559 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514561 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514564 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514567 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514569 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514571 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514575 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514577 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:37.520975 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514580 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514583 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.514588 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514718 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514724 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514727 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514730 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514733 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514736 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514738 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514741 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514743 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514746 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514749 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514751 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:08:37.521461 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514754 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514757 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514759 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514762 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514764 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514767 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514769 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514771 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514774 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514777 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514779 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514782 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514784 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514787 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514789 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514792 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514795 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514798 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514801 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514803 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:08:37.521978 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514806 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514808 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514811 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514813 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514816 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514818 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514821 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514823 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514838 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514840 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514843 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514846 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514849 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514852 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514855 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514859 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514863 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514866 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514869 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514872 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:08:37.522502 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514875 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514877 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514880 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514883 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514888 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514891 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514894 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514898 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514901 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514904 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514907 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514909 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514911 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514914 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514917 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514919 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514922 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514924 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514927 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514930 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:08:37.523025 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514932 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514935 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514937 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514940 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514943 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514945 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514948 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514951 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514954 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514958 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514961 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514963 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514967 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514971 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514974 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514977 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514981 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514984 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:08:37.523725 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514987 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514991 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514994 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.514997 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.515000 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.515002 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.515005 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.515008 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:37.515010 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.515015 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.515168 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.517371 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.518807 2570 server.go:1019] "Starting client certificate rotation" Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.518930 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:08:37.524446 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.518985 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:08:37.525515 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.525230 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:08:37.529185 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.529155 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:08:37.540015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.539994 2570 log.go:25] "Validated CRI v1 runtime API" Jul 28 17:08:37.542130 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.542107 2570 log.go:25] "Validated CRI v1 image API" Jul 28 17:08:37.543753 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.543732 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 17:08:37.546887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.546866 2570 fs.go:135] Filesystem UUIDs: map[304b15a9-9bd8-4b18-b639-1dceb7739e2a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cbe19266-7ad7-4628-b809-4b1e2e07ff2d:/dev/nvme0n1p3] Jul 28 17:08:37.546950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.546885 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 17:08:37.553174 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.553070 2570 manager.go:216] Machine: {Timestamp:2026-07-28 17:08:37.550867996 +0000 UTC m=+0.309728904 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099260 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec271f0bb7f636eb7be32413b0980730 SystemUUID:ec271f0b-b7f6-36eb-7be3-2413b0980730 BootID:65bd857e-18e9-4d7c-8ce3-f230323a62dc Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e7:7d:19:de:49 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e7:7d:19:de:49 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:22:06:15:8e:09 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 17:08:37.553174 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.553170 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 17:08:37.553294 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.553254 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 17:08:37.556042 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.556015 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 17:08:37.556209 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.556045 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-215.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 17:08:37.556775 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.556766 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 17:08:37.556813 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.556778 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 17:08:37.556813 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.556786 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:08:37.557283 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.557273 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:08:37.557705 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.557689 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:08:37.558512 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.558499 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 17:08:37.561210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561198 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:08:37.561322 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561313 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 17:08:37.561360 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561344 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 28 17:08:37.561360 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561359 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 17:08:37.561412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561374 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 17:08:37.561412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561383 2570 kubelet.go:398] "Adding apiserver pod source" Jul 28 17:08:37.561412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561395 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 17:08:37.561907 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561894 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:08:37.561944 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.561923 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:08:37.563909 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.563893 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 17:08:37.565816 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.565795 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 17:08:37.565871 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.565862 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 17:08:37.567985 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.567965 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 17:08:37.568074 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.567998 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 17:08:37.568074 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568010 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 17:08:37.568074 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568019 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 17:08:37.568074 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568033 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 17:08:37.568074 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568042 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 17:08:37.568074 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568062 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 17:08:37.568074 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568071 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 17:08:37.568390 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568095 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 17:08:37.568390 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568104 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 17:08:37.568390 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568132 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 17:08:37.568390 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568141 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 17:08:37.568742 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568726 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 17:08:37.568742 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.568745 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 17:08:37.572493 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.572474 2570 server.go:1268] "Started kubelet" Jul 28 17:08:37.572599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.572556 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 17:08:37.572651 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.572555 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 17:08:37.572651 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.572634 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 17:08:37.573472 ip-10-0-136-215 systemd[1]: Started Kubernetes Kubelet. Jul 28 17:08:37.574325 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.574303 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 17:08:37.574447 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.574434 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 28 17:08:37.574863 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.574823 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 17:08:37.574971 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.574883 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-215.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 17:08:37.575065 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.574923 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 17:08:37.577365 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.577339 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 17:08:37.579820 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.578939 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c68329bc254f04 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-28 17:08:37.572439812 +0000 UTC m=+0.331300716,LastTimestamp:2026-07-28 17:08:37.572439812 +0000 UTC m=+0.331300716,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 28 17:08:37.581458 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.581438 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 17:08:37.581458 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.581453 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 17:08:37.581594 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.581520 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:37.582045 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582027 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 17:08:37.582045 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582036 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 17:08:37.582198 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582051 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 17:08:37.582245 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582200 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 28 17:08:37.582245 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582208 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 28 17:08:37.582464 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582447 2570 factory.go:55] Registering systemd factory Jul 28 17:08:37.582464 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582464 2570 factory.go:223] Registration of the systemd container factory successfully Jul 28 17:08:37.582681 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582668 2570 factory.go:153] Registering CRI-O factory Jul 28 17:08:37.582739 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582685 2570 factory.go:223] Registration of the crio container factory successfully Jul 28 17:08:37.582739 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582733 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 17:08:37.582866 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582752 2570 factory.go:103] Registering Raw factory Jul 28 17:08:37.582866 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.582764 2570 manager.go:1196] Started watching for new ooms in manager Jul 28 17:08:37.583245 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.583233 2570 manager.go:349] Starting recovery of all containers Jul 28 17:08:37.588772 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.588745 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-215.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 17:08:37.588902 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.588874 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 17:08:37.594644 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.593793 2570 manager.go:354] Recovery completed Jul 28 17:08:37.598871 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.598854 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:37.600773 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.600756 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:37.600867 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.600787 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:37.600867 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.600798 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:37.601454 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.601359 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 17:08:37.601454 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.601369 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 17:08:37.601454 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.601385 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:08:37.601454 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.601367 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:37.602298 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.602281 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:37.602384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.602310 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:37.602384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.602325 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:37.603354 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.603336 2570 policy_none.go:49] "None policy: Start" Jul 28 17:08:37.603419 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.603361 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 17:08:37.603419 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.603376 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 17:08:37.603419 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.603300 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c68329bdd5a620 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-28 17:08:37.600773664 +0000 UTC m=+0.359634568,LastTimestamp:2026-07-28 17:08:37.600773664 +0000 UTC m=+0.359634568,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 28 17:08:37.604604 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.604588 2570 policy_none.go:47] "Start" Jul 28 17:08:37.609550 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.609535 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2j5tj" Jul 28 17:08:37.611855 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.611759 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c68329bdd5ee93 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-136-215.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-28 17:08:37.600792211 +0000 UTC m=+0.359653116,LastTimestamp:2026-07-28 17:08:37.600792211 +0000 UTC m=+0.359653116,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 28 17:08:37.614399 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.614315 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c68329bdd615eb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-136-215.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-28 17:08:37.600802283 +0000 UTC m=+0.359663189,LastTimestamp:2026-07-28 17:08:37.600802283 +0000 UTC m=+0.359663189,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 28 17:08:37.619414 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.619284 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2j5tj" Jul 28 17:08:37.621994 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.621919 2570 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-136-215.ec2.internal.18c68329bdd5a620\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-215.ec2.internal.18c68329bdd5a620 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-215.ec2.internal,UID:ip-10-0-136-215.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-215.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-215.ec2.internal,},FirstTimestamp:2026-07-28 17:08:37.600773664 +0000 UTC m=+0.359634568,LastTimestamp:2026-07-28 17:08:37.602295766 +0000 UTC m=+0.361156670,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-215.ec2.internal,}" Jul 28 17:08:37.630517 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.630501 2570 manager.go:337] "Starting Device Plugin manager" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.630538 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.630549 2570 server.go:87] "Starting device plugin registration server" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.630897 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.630917 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.631313 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.631402 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.631411 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.631756 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 17:08:37.645652 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.631819 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:37.715655 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.715611 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 17:08:37.716917 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.716897 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 17:08:37.716988 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.716945 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 17:08:37.716988 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.716978 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 17:08:37.717078 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.717022 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 17:08:37.719193 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.719171 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:37.731767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.731740 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:37.732697 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.732680 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:37.732782 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.732710 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:37.732782 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.732720 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:37.732782 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.732750 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.739794 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.739771 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.739794 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.739794 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-215.ec2.internal\": node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:37.768156 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.768087 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:37.817687 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.817642 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal"] Jul 28 17:08:37.817872 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.817732 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:37.818715 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.818699 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:37.818788 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.818730 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:37.818788 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.818742 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:37.820032 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820018 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:37.820164 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820147 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.820204 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820195 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:37.820755 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820735 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:37.820853 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820767 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:37.820853 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820777 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:37.820853 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820738 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:37.820953 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820854 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:37.820953 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.820867 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:37.821884 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.821867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.821982 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.821891 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:37.822553 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.822530 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:37.822623 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.822574 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:37.822623 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.822591 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:37.845684 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.845657 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-215.ec2.internal\" not found" node="ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.850189 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.850170 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-215.ec2.internal\" not found" node="ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.868340 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.868300 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:37.884366 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.884338 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4746d2d48c5a0af4bf6819c4fa3b9ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"4746d2d48c5a0af4bf6819c4fa3b9ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.884465 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.884381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.884465 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.884415 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.969465 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:37.969440 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:37.984805 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.984781 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4746d2d48c5a0af4bf6819c4fa3b9ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"4746d2d48c5a0af4bf6819c4fa3b9ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.984880 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.984813 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.984880 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.984846 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.984948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.984876 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4746d2d48c5a0af4bf6819c4fa3b9ae9-config\") pod \"kube-apiserver-proxy-ip-10-0-136-215.ec2.internal\" (UID: \"4746d2d48c5a0af4bf6819c4fa3b9ae9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.984948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.984894 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:37.984948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:37.984903 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42785eed679f9257ac8637d2df48ef77-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal\" (UID: \"42785eed679f9257ac8637d2df48ef77\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:38.070293 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.070213 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.147727 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.147694 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:38.153504 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.153093 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:08:38.171374 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.171338 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.271917 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.271877 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.372468 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.372440 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.428653 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.428626 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:38.473562 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.473524 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.518132 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.518100 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 17:08:38.518576 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.518273 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:08:38.518576 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.518273 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:08:38.573706 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.573674 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.624614 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.624513 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:03:38 +0000 UTC" deadline="2028-02-22 17:05:42.343046147 +0000 UTC" Jul 28 17:08:38.624614 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.624553 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13775h57m3.718497349s" Jul 28 17:08:38.674132 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.674099 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.699940 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:38.699890 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4746d2d48c5a0af4bf6819c4fa3b9ae9.slice/crio-a93e779289a5c78298916fd776241f5c2614e2ea881e94aec65f77f3b7cd9d3f WatchSource:0}: Error finding container a93e779289a5c78298916fd776241f5c2614e2ea881e94aec65f77f3b7cd9d3f: Status 404 returned error can't find the container with id a93e779289a5c78298916fd776241f5c2614e2ea881e94aec65f77f3b7cd9d3f Jul 28 17:08:38.700131 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:38.700109 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42785eed679f9257ac8637d2df48ef77.slice/crio-b7c803f9f124956392837ad970977538775948ab25e0f5864ae82aa8f2c4a1c7 WatchSource:0}: Error finding container b7c803f9f124956392837ad970977538775948ab25e0f5864ae82aa8f2c4a1c7: Status 404 returned error can't find the container with id b7c803f9f124956392837ad970977538775948ab25e0f5864ae82aa8f2c4a1c7 Jul 28 17:08:38.701220 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.701206 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:08:38.702610 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.702495 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:08:38.702610 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.702535 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:08:38.702610 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.702545 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:08:38.702610 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.702553 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:38.705032 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.705014 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:08:38.720486 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.720434 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" event={"ID":"4746d2d48c5a0af4bf6819c4fa3b9ae9","Type":"ContainerStarted","Data":"a93e779289a5c78298916fd776241f5c2614e2ea881e94aec65f77f3b7cd9d3f"} Jul 28 17:08:38.721681 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.721657 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"42785eed679f9257ac8637d2df48ef77","Type":"ContainerStarted","Data":"b7c803f9f124956392837ad970977538775948ab25e0f5864ae82aa8f2c4a1c7"} Jul 28 17:08:38.774738 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:38.774692 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-215.ec2.internal\" not found" Jul 28 17:08:38.859464 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.859433 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:38.882083 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.882010 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" Jul 28 17:08:38.898714 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.898688 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:08:38.899689 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.899673 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" Jul 28 17:08:38.911770 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:38.911747 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:08:39.563338 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.563308 2570 apiserver.go:52] "Watching apiserver" Jul 28 17:08:39.571201 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.571172 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 17:08:39.571673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.571648 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-kcdsz","kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal","openshift-dns/node-resolver-vdrdq","openshift-image-registry/node-ca-vbj6k","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal","openshift-multus/multus-additional-cni-plugins-c9hbp","openshift-multus/multus-t9m7d","openshift-multus/network-metrics-daemon-plwl2","kube-system/global-pull-secret-syncer-62dsj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg","openshift-cluster-node-tuning-operator/tuned-d92lh","openshift-network-diagnostics/network-check-target-qtjtr","openshift-network-operator/iptables-alerter-tp7cx","openshift-ovn-kubernetes/ovnkube-node-r9ndn"] Jul 28 17:08:39.573549 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.573407 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.575437 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.575089 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.576603 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.576421 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.577367 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.576853 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 17:08:39.577367 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.576987 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 17:08:39.577367 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.577085 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 17:08:39.577367 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.577136 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 17:08:39.577367 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.577240 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-7nsxk\"" Jul 28 17:08:39.579210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.577992 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:39.579210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.578255 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:39.579210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.578458 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xxbdm\"" Jul 28 17:08:39.579210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.579000 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.579953 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.579759 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.580321 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.580205 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 17:08:39.581432 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.581407 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 17:08:39.581701 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.581682 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ctzqn\"" Jul 28 17:08:39.582040 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.582021 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 17:08:39.582134 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.582022 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:08:39.582134 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.582126 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583283 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.583359 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583614 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583667 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583786 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583797 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583916 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wswlp\"" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583962 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 17:08:39.584129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.583991 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 17:08:39.585083 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.584642 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 17:08:39.585083 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.584656 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qqvtv\"" Jul 28 17:08:39.585776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.585754 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 17:08:39.588265 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.586610 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.588265 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.587639 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 17:08:39.588415 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.588396 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 17:08:39.589051 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.589030 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5q9fn\"" Jul 28 17:08:39.589335 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.589317 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:39.589516 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.589466 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:39.589779 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.589760 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:08:39.590315 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.590107 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 17:08:39.591546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.590509 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.591546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.590574 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 17:08:39.591546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.590507 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bgvtz\"" Jul 28 17:08:39.592910 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.592779 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594015 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-host\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594076 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e93e778-0d32-4bf2-8be0-7999c020ae7e-tmp\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594098 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594122 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-ovnkube-script-lib\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594145 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.594208 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:39.594248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27nts\" (UniqueName: \"kubernetes.io/projected/48646f83-5284-4e24-8ea7-da85a74922aa-kube-api-access-27nts\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594270 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-system-cni-dir\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594299 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-multus-certs\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594326 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-system-cni-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594351 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-conf-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594374 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-node-log\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594397 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-cni-bin\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-kubelet\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594444 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-var-lib-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594470 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-etc-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594493 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-tuned\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-os-release\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594643 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-cni-bin\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594671 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594699 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-cnibin\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.594887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594725 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e569df22-dae2-4b3d-b50a-6fda9da8140b-cni-binary-copy\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-netns\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594792 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cfd4\" (UniqueName: \"kubernetes.io/projected/987609de-891a-47cb-b2da-262a39bda2fa-kube-api-access-9cfd4\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594828 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594872 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l55gc\" (UniqueName: \"kubernetes.io/projected/e569df22-dae2-4b3d-b50a-6fda9da8140b-kube-api-access-l55gc\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594903 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-systemd-units\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594960 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.594999 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-kubelet\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595059 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-hostroot\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595137 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595231 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595089 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klkfr\" (UniqueName: \"kubernetes.io/projected/90d0c9d9-154c-4867-acd4-3c4a14069201-kube-api-access-klkfr\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595332 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-run\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595362 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-cni-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595391 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-cni-multus\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595415 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-slash\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595439 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-var-lib-kubelet\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595462 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5feed6d1-3fca-42ac-991a-4803dbe46b52-dbus\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595481 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 17:08:39.595993 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595511 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595504 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2st6\" (UniqueName: \"kubernetes.io/projected/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-kube-api-access-q2st6\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595620 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8lv9m\"" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595643 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-socket-dir-parent\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595670 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-daemon-config\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595698 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-os-release\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-serviceca\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595752 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-systemd\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595777 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-cni-netd\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595801 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-env-overrides\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-cnibin\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595882 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysconfig\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595923 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-sys\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-ovn\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595969 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.595991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-ovnkube-config\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596018 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/987609de-891a-47cb-b2da-262a39bda2fa-ovn-node-metrics-cert\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596036 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ksmbc\"" Jul 28 17:08:39.596948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596048 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-kubernetes\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596072 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-systemd\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596097 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h68h7\" (UniqueName: \"kubernetes.io/projected/8e93e778-0d32-4bf2-8be0-7999c020ae7e-kube-api-access-h68h7\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5feed6d1-3fca-42ac-991a-4803dbe46b52-kubelet-config\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596150 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-etc-kubernetes\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596202 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-run-netns\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596247 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-lib-modules\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596328 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/48646f83-5284-4e24-8ea7-da85a74922aa-hosts-file\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596387 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596414 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-host\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-log-socket\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596462 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-modprobe-d\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596496 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysctl-d\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596551 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysctl-conf\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596584 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48646f83-5284-4e24-8ea7-da85a74922aa-tmp-dir\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.597789 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.596623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-k8s-cni-cncf-io\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.601252 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.601229 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:08:39.632642 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.632620 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-sz6fv" Jul 28 17:08:39.643208 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.643190 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:39.647658 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.647632 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-sz6fv" Jul 28 17:08:39.683853 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.683822 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 17:08:39.697510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-cnibin\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.697510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697507 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e569df22-dae2-4b3d-b50a-6fda9da8140b-cni-binary-copy\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697532 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-netns\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697552 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cfd4\" (UniqueName: \"kubernetes.io/projected/987609de-891a-47cb-b2da-262a39bda2fa-kube-api-access-9cfd4\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697577 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697600 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-cnibin\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697608 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxwdf\" (UniqueName: \"kubernetes.io/projected/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-kube-api-access-dxwdf\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697657 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697686 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l55gc\" (UniqueName: \"kubernetes.io/projected/e569df22-dae2-4b3d-b50a-6fda9da8140b-kube-api-access-l55gc\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-systemd-units\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.697748 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697733 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697772 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697780 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-socket-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697611 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-netns\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-iptables-alerter-script\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697848 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-kubelet\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697892 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-hostroot\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klkfr\" (UniqueName: \"kubernetes.io/projected/90d0c9d9-154c-4867-acd4-3c4a14069201-kube-api-access-klkfr\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697947 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-run\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697972 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-cni-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.697992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-cni-multus\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698008 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-slash\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698066 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-systemd-units\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698077 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-hostroot\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698077 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-kubelet\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698109 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-slash\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.698236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698140 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-var-lib-kubelet\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698187 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-cni-multus\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698195 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-run\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698196 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5feed6d1-3fca-42ac-991a-4803dbe46b52-dbus\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698237 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-var-lib-kubelet\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698274 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2st6\" (UniqueName: \"kubernetes.io/projected/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-kube-api-access-q2st6\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698261 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e569df22-dae2-4b3d-b50a-6fda9da8140b-cni-binary-copy\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-socket-dir-parent\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698344 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5feed6d1-3fca-42ac-991a-4803dbe46b52-dbus\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698348 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-cni-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698429 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-daemon-config\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698482 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-os-release\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698466 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-socket-dir-parent\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698501 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-serviceca\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698520 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-systemd\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698573 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-systemd\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698582 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-registration-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.699099 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698606 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-os-release\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698615 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-device-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698660 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-cni-netd\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.698758 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-cni-netd\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699035 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-daemon-config\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699358 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-env-overrides\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699431 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-cnibin\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699462 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysconfig\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699491 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-sys\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699515 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-cnibin\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699521 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-sys-fs\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699544 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysconfig\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699556 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a9b8086-4250-4feb-baee-b561b9015044-agent-certs\") pod \"konnectivity-agent-kcdsz\" (UID: \"2a9b8086-4250-4feb-baee-b561b9015044\") " pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699580 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-sys\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699595 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b8086-4250-4feb-baee-b561b9015044-konnectivity-ca\") pod \"konnectivity-agent-kcdsz\" (UID: \"2a9b8086-4250-4feb-baee-b561b9015044\") " pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699627 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-ovn\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699669 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-ovnkube-config\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699684 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-serviceca\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/987609de-891a-47cb-b2da-262a39bda2fa-ovn-node-metrics-cert\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699672 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-ovn\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-kubernetes\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699777 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-env-overrides\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699892 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-systemd\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699899 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-kubernetes\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699945 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h68h7\" (UniqueName: \"kubernetes.io/projected/8e93e778-0d32-4bf2-8be0-7999c020ae7e-kube-api-access-h68h7\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699961 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-systemd\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.699999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5feed6d1-3fca-42ac-991a-4803dbe46b52-kubelet-config\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700098 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-etc-kubernetes\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-run-netns\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700144 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5feed6d1-3fca-42ac-991a-4803dbe46b52-kubelet-config\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700144 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700167 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700192 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-lib-modules\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700198 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-ovnkube-config\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700211 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-etc-kubernetes\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.700829 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700215 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/48646f83-5284-4e24-8ea7-da85a74922aa-hosts-file\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700230 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-run-netns\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700258 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-run-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-host\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700271 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/48646f83-5284-4e24-8ea7-da85a74922aa-hosts-file\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700306 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-log-socket\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700320 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-host\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.700330 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-modprobe-d\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700350 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-log-socket\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700358 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysctl-d\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700323 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-lib-modules\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.700412 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret podName:5feed6d1-3fca-42ac-991a-4803dbe46b52 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:40.200372255 +0000 UTC m=+2.959233170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret") pod "global-pull-secret-syncer-62dsj" (UID: "5feed6d1-3fca-42ac-991a-4803dbe46b52") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700447 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-modprobe-d\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700466 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysctl-d\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700478 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysctl-conf\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.701776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700505 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48646f83-5284-4e24-8ea7-da85a74922aa-tmp-dir\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-k8s-cni-cncf-io\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700615 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-host\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700633 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e93e778-0d32-4bf2-8be0-7999c020ae7e-tmp\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700616 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-k8s-cni-cncf-io\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700648 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-sysctl-conf\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700655 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-host-slash\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700681 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8e93e778-0d32-4bf2-8be0-7999c020ae7e-host\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-ovnkube-script-lib\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700774 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27nts\" (UniqueName: \"kubernetes.io/projected/48646f83-5284-4e24-8ea7-da85a74922aa-kube-api-access-27nts\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgj77\" (UniqueName: \"kubernetes.io/projected/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-kube-api-access-sgj77\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700774 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700837 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48646f83-5284-4e24-8ea7-da85a74922aa-tmp-dir\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.702634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700875 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-system-cni-dir\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700905 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-multus-certs\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700930 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-system-cni-dir\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-system-cni-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700965 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-conf-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-node-log\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.700990 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-run-multus-certs\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701018 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-system-cni-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701031 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-multus-conf-dir\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701055 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k8d8\" (UniqueName: \"kubernetes.io/projected/0638b35e-71d3-4b3f-b998-aea01f04aee1-kube-api-access-4k8d8\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-cni-bin\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701070 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-node-log\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701100 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-kubelet\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701132 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e569df22-dae2-4b3d-b50a-6fda9da8140b-host-var-lib-cni-bin\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701142 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-kubelet\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701154 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-var-lib-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701179 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-etc-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.703555 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701110 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701198 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-var-lib-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701200 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-tuned\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701240 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-os-release\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701245 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-etc-openvswitch\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701254 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/987609de-891a-47cb-b2da-262a39bda2fa-ovnkube-script-lib\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701301 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-cni-bin\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701325 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/90d0c9d9-154c-4867-acd4-3c4a14069201-os-release\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701330 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701371 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701390 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/987609de-891a-47cb-b2da-262a39bda2fa-host-cni-bin\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.704412 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.701736 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/90d0c9d9-154c-4867-acd4-3c4a14069201-cni-binary-copy\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.704933 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.704491 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e93e778-0d32-4bf2-8be0-7999c020ae7e-tmp\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.704933 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.704584 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8e93e778-0d32-4bf2-8be0-7999c020ae7e-etc-tuned\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.704933 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.704701 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/987609de-891a-47cb-b2da-262a39bda2fa-ovn-node-metrics-cert\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.715983 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.715955 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27nts\" (UniqueName: \"kubernetes.io/projected/48646f83-5284-4e24-8ea7-da85a74922aa-kube-api-access-27nts\") pod \"node-resolver-vdrdq\" (UID: \"48646f83-5284-4e24-8ea7-da85a74922aa\") " pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.716121 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.716023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klkfr\" (UniqueName: \"kubernetes.io/projected/90d0c9d9-154c-4867-acd4-3c4a14069201-kube-api-access-klkfr\") pod \"multus-additional-cni-plugins-c9hbp\" (UID: \"90d0c9d9-154c-4867-acd4-3c4a14069201\") " pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.716215 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.716190 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2st6\" (UniqueName: \"kubernetes.io/projected/ba17a781-9e7c-4ec2-ab60-9a2d4132c464-kube-api-access-q2st6\") pod \"node-ca-vbj6k\" (UID: \"ba17a781-9e7c-4ec2-ab60-9a2d4132c464\") " pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.716950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.716923 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l55gc\" (UniqueName: \"kubernetes.io/projected/e569df22-dae2-4b3d-b50a-6fda9da8140b-kube-api-access-l55gc\") pod \"multus-t9m7d\" (UID: \"e569df22-dae2-4b3d-b50a-6fda9da8140b\") " pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.718956 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.718932 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cfd4\" (UniqueName: \"kubernetes.io/projected/987609de-891a-47cb-b2da-262a39bda2fa-kube-api-access-9cfd4\") pod \"ovnkube-node-r9ndn\" (UID: \"987609de-891a-47cb-b2da-262a39bda2fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.723578 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.721354 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h68h7\" (UniqueName: \"kubernetes.io/projected/8e93e778-0d32-4bf2-8be0-7999c020ae7e-kube-api-access-h68h7\") pod \"tuned-d92lh\" (UID: \"8e93e778-0d32-4bf2-8be0-7999c020ae7e\") " pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.801843 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801804 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801864 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-host-slash\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.802012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801918 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgj77\" (UniqueName: \"kubernetes.io/projected/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-kube-api-access-sgj77\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.802012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801924 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-host-slash\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.802012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801947 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4k8d8\" (UniqueName: \"kubernetes.io/projected/0638b35e-71d3-4b3f-b998-aea01f04aee1-kube-api-access-4k8d8\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxwdf\" (UniqueName: \"kubernetes.io/projected/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-kube-api-access-dxwdf\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:39.802012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802009 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.801940 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802037 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802069 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-etc-selinux\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-socket-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.802112 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802131 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-iptables-alerter-script\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.802167 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:40.30214628 +0000 UTC m=+3.061007174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802203 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-socket-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802206 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-registration-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802241 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-device-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802255 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-registration-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802262 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-sys-fs\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a9b8086-4250-4feb-baee-b561b9015044-agent-certs\") pod \"konnectivity-agent-kcdsz\" (UID: \"2a9b8086-4250-4feb-baee-b561b9015044\") " pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802300 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-sys-fs\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.802384 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802342 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b8086-4250-4feb-baee-b561b9015044-konnectivity-ca\") pod \"konnectivity-agent-kcdsz\" (UID: \"2a9b8086-4250-4feb-baee-b561b9015044\") " pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:39.803094 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802415 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0638b35e-71d3-4b3f-b998-aea01f04aee1-device-dir\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.803094 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802767 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-iptables-alerter-script\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.803094 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.802974 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2a9b8086-4250-4feb-baee-b561b9015044-konnectivity-ca\") pod \"konnectivity-agent-kcdsz\" (UID: \"2a9b8086-4250-4feb-baee-b561b9015044\") " pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:39.805027 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.805005 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2a9b8086-4250-4feb-baee-b561b9015044-agent-certs\") pod \"konnectivity-agent-kcdsz\" (UID: \"2a9b8086-4250-4feb-baee-b561b9015044\") " pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:39.818060 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.817990 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgj77\" (UniqueName: \"kubernetes.io/projected/cc70882a-a450-4afd-9a42-8eaa3bcc61d6-kube-api-access-sgj77\") pod \"iptables-alerter-tp7cx\" (UID: \"cc70882a-a450-4afd-9a42-8eaa3bcc61d6\") " pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.828655 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.828622 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxwdf\" (UniqueName: \"kubernetes.io/projected/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-kube-api-access-dxwdf\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:39.829599 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.829565 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:39.829599 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.829593 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:39.829792 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.829607 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qlv9x for pod openshift-network-diagnostics/network-check-target-qtjtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:39.829792 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:39.829687 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x podName:93cfcf12-af34-48a1-81dd-4957742144ff nodeName:}" failed. No retries permitted until 2026-07-28 17:08:40.329666332 +0000 UTC m=+3.088527238 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qlv9x" (UniqueName: "kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x") pod "network-check-target-qtjtr" (UID: "93cfcf12-af34-48a1-81dd-4957742144ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:39.832135 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.832114 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k8d8\" (UniqueName: \"kubernetes.io/projected/0638b35e-71d3-4b3f-b998-aea01f04aee1-kube-api-access-4k8d8\") pod \"aws-ebs-csi-driver-node-9v8vg\" (UID: \"0638b35e-71d3-4b3f-b998-aea01f04aee1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.888874 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.888818 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t9m7d" Jul 28 17:08:39.898630 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.898602 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-d92lh" Jul 28 17:08:39.908090 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.908065 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vdrdq" Jul 28 17:08:39.914772 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.914745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vbj6k" Jul 28 17:08:39.924165 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.924136 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:08:39.931038 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.931015 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" Jul 28 17:08:39.938795 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.938772 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tp7cx" Jul 28 17:08:39.944127 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.944106 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" Jul 28 17:08:39.949935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:39.949913 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:08:40.204712 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.204674 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:40.204939 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.204815 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:40.204939 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.204907 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret podName:5feed6d1-3fca-42ac-991a-4803dbe46b52 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:41.204882479 +0000 UTC m=+3.963743377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret") pod "global-pull-secret-syncer-62dsj" (UID: "5feed6d1-3fca-42ac-991a-4803dbe46b52") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:40.305615 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.305571 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:40.305821 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.305768 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:40.305903 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.305867 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:41.305830918 +0000 UTC m=+4.064691814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:40.406914 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.406872 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:40.407087 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.407063 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:40.407128 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.407099 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:40.407128 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.407117 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qlv9x for pod openshift-network-diagnostics/network-check-target-qtjtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:40.407211 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:40.407192 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x podName:93cfcf12-af34-48a1-81dd-4957742144ff nodeName:}" failed. No retries permitted until 2026-07-28 17:08:41.407171037 +0000 UTC m=+4.166031958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlv9x" (UniqueName: "kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x") pod "network-check-target-qtjtr" (UID: "93cfcf12-af34-48a1-81dd-4957742144ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:40.649411 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.649339 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:03:40 +0000 UTC" deadline="2028-04-09 17:51:50.576800364 +0000 UTC" Jul 28 17:08:40.649411 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.649404 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14904h43m9.927399371s" Jul 28 17:08:40.687877 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.687742 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e93e778_0d32_4bf2_8be0_7999c020ae7e.slice/crio-021dfa3d3e29f0ba06305030a166642d89ab5f253501703352f52375030c6de8 WatchSource:0}: Error finding container 021dfa3d3e29f0ba06305030a166642d89ab5f253501703352f52375030c6de8: Status 404 returned error can't find the container with id 021dfa3d3e29f0ba06305030a166642d89ab5f253501703352f52375030c6de8 Jul 28 17:08:40.689813 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.689660 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc70882a_a450_4afd_9a42_8eaa3bcc61d6.slice/crio-95557a55a96ce643d40c9bfcfa25f41405a6175f4af359b9630d5fd15b2f9292 WatchSource:0}: Error finding container 95557a55a96ce643d40c9bfcfa25f41405a6175f4af359b9630d5fd15b2f9292: Status 404 returned error can't find the container with id 95557a55a96ce643d40c9bfcfa25f41405a6175f4af359b9630d5fd15b2f9292 Jul 28 17:08:40.692826 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.692805 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode569df22_dae2_4b3d_b50a_6fda9da8140b.slice/crio-b1da344697c1bc4bb927b577ed013d16b6393e17756c3d3cc019771f74d5e8f8 WatchSource:0}: Error finding container b1da344697c1bc4bb927b577ed013d16b6393e17756c3d3cc019771f74d5e8f8: Status 404 returned error can't find the container with id b1da344697c1bc4bb927b577ed013d16b6393e17756c3d3cc019771f74d5e8f8 Jul 28 17:08:40.694100 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.694076 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987609de_891a_47cb_b2da_262a39bda2fa.slice/crio-ea64bf924b25289078a763f4fb148c26151b89d8690ea99e617b81b3b1dd6ccd WatchSource:0}: Error finding container ea64bf924b25289078a763f4fb148c26151b89d8690ea99e617b81b3b1dd6ccd: Status 404 returned error can't find the container with id ea64bf924b25289078a763f4fb148c26151b89d8690ea99e617b81b3b1dd6ccd Jul 28 17:08:40.695447 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.695427 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90d0c9d9_154c_4867_acd4_3c4a14069201.slice/crio-26d709a525bc70c05aec8e9a3b430baafffb62c0d9ec3dc01c41bd31d7de8a80 WatchSource:0}: Error finding container 26d709a525bc70c05aec8e9a3b430baafffb62c0d9ec3dc01c41bd31d7de8a80: Status 404 returned error can't find the container with id 26d709a525bc70c05aec8e9a3b430baafffb62c0d9ec3dc01c41bd31d7de8a80 Jul 28 17:08:40.696961 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.696934 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48646f83_5284_4e24_8ea7_da85a74922aa.slice/crio-5d3836856684228cb63bb3d1841632fc7dc36cf4ebc2f416dec98dd2d728a88e WatchSource:0}: Error finding container 5d3836856684228cb63bb3d1841632fc7dc36cf4ebc2f416dec98dd2d728a88e: Status 404 returned error can't find the container with id 5d3836856684228cb63bb3d1841632fc7dc36cf4ebc2f416dec98dd2d728a88e Jul 28 17:08:40.697815 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.697790 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0638b35e_71d3_4b3f_b998_aea01f04aee1.slice/crio-04dd9b4356b73aa7bbd6c36c057687c8f02c2f87e110ea0e543f9cfecf04c0e5 WatchSource:0}: Error finding container 04dd9b4356b73aa7bbd6c36c057687c8f02c2f87e110ea0e543f9cfecf04c0e5: Status 404 returned error can't find the container with id 04dd9b4356b73aa7bbd6c36c057687c8f02c2f87e110ea0e543f9cfecf04c0e5 Jul 28 17:08:40.699932 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.699828 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a9b8086_4250_4feb_baee_b561b9015044.slice/crio-74630012f39f7c559c0c36f78dfd76e7958200959a83bcf7c9346f1b5a383593 WatchSource:0}: Error finding container 74630012f39f7c559c0c36f78dfd76e7958200959a83bcf7c9346f1b5a383593: Status 404 returned error can't find the container with id 74630012f39f7c559c0c36f78dfd76e7958200959a83bcf7c9346f1b5a383593 Jul 28 17:08:40.701697 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:08:40.701613 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba17a781_9e7c_4ec2_ab60_9a2d4132c464.slice/crio-f1d11f3f46268ad0efabf0cb0de9b079a5877eb38d67cad76ca823c7eb292577 WatchSource:0}: Error finding container f1d11f3f46268ad0efabf0cb0de9b079a5877eb38d67cad76ca823c7eb292577: Status 404 returned error can't find the container with id f1d11f3f46268ad0efabf0cb0de9b079a5877eb38d67cad76ca823c7eb292577 Jul 28 17:08:40.707150 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.707119 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:08:40.728290 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.728255 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kcdsz" event={"ID":"2a9b8086-4250-4feb-baee-b561b9015044","Type":"ContainerStarted","Data":"74630012f39f7c559c0c36f78dfd76e7958200959a83bcf7c9346f1b5a383593"} Jul 28 17:08:40.729603 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.729573 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerStarted","Data":"26d709a525bc70c05aec8e9a3b430baafffb62c0d9ec3dc01c41bd31d7de8a80"} Jul 28 17:08:40.730739 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.730708 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"ea64bf924b25289078a763f4fb148c26151b89d8690ea99e617b81b3b1dd6ccd"} Jul 28 17:08:40.732975 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.732950 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d92lh" event={"ID":"8e93e778-0d32-4bf2-8be0-7999c020ae7e","Type":"ContainerStarted","Data":"021dfa3d3e29f0ba06305030a166642d89ab5f253501703352f52375030c6de8"} Jul 28 17:08:40.735114 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.735092 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vbj6k" event={"ID":"ba17a781-9e7c-4ec2-ab60-9a2d4132c464","Type":"ContainerStarted","Data":"f1d11f3f46268ad0efabf0cb0de9b079a5877eb38d67cad76ca823c7eb292577"} Jul 28 17:08:40.740034 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.740005 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" event={"ID":"0638b35e-71d3-4b3f-b998-aea01f04aee1","Type":"ContainerStarted","Data":"04dd9b4356b73aa7bbd6c36c057687c8f02c2f87e110ea0e543f9cfecf04c0e5"} Jul 28 17:08:40.741415 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.741388 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vdrdq" event={"ID":"48646f83-5284-4e24-8ea7-da85a74922aa","Type":"ContainerStarted","Data":"5d3836856684228cb63bb3d1841632fc7dc36cf4ebc2f416dec98dd2d728a88e"} Jul 28 17:08:40.742766 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.742736 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t9m7d" event={"ID":"e569df22-dae2-4b3d-b50a-6fda9da8140b","Type":"ContainerStarted","Data":"b1da344697c1bc4bb927b577ed013d16b6393e17756c3d3cc019771f74d5e8f8"} Jul 28 17:08:40.743895 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:40.743824 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tp7cx" event={"ID":"cc70882a-a450-4afd-9a42-8eaa3bcc61d6","Type":"ContainerStarted","Data":"95557a55a96ce643d40c9bfcfa25f41405a6175f4af359b9630d5fd15b2f9292"} Jul 28 17:08:41.214196 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.213683 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:41.214196 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.214045 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:41.214196 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.214130 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret podName:5feed6d1-3fca-42ac-991a-4803dbe46b52 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:43.214107105 +0000 UTC m=+5.972968005 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret") pod "global-pull-secret-syncer-62dsj" (UID: "5feed6d1-3fca-42ac-991a-4803dbe46b52") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:41.315098 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.315057 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:41.315286 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.315234 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:41.315355 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.315310 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:43.315289401 +0000 UTC m=+6.074150294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:41.415650 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.415601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:41.415861 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.415805 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:41.415861 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.415828 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:41.415861 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.415842 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qlv9x for pod openshift-network-diagnostics/network-check-target-qtjtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:41.416040 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.415935 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x podName:93cfcf12-af34-48a1-81dd-4957742144ff nodeName:}" failed. No retries permitted until 2026-07-28 17:08:43.415906344 +0000 UTC m=+6.174767261 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlv9x" (UniqueName: "kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x") pod "network-check-target-qtjtr" (UID: "93cfcf12-af34-48a1-81dd-4957742144ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:41.650505 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.650409 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:03:40 +0000 UTC" deadline="2028-02-25 09:39:44.329506315 +0000 UTC" Jul 28 17:08:41.650505 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.650473 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13840h31m2.679037892s" Jul 28 17:08:41.720754 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.720650 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:41.720965 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.720803 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:41.721396 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.721373 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:41.721504 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.721481 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:41.721590 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.721577 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:41.721677 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:41.721660 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:41.761586 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.761545 2570 generic.go:356] "Generic (PLEG): container finished" podID="42785eed679f9257ac8637d2df48ef77" containerID="c898f4272b8222a7e635ad4e4a08383949eb91ebb97c82a84bed700f6cabac42" exitCode=0 Jul 28 17:08:41.761775 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.761654 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"42785eed679f9257ac8637d2df48ef77","Type":"ContainerDied","Data":"c898f4272b8222a7e635ad4e4a08383949eb91ebb97c82a84bed700f6cabac42"} Jul 28 17:08:41.779286 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:41.779241 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" event={"ID":"4746d2d48c5a0af4bf6819c4fa3b9ae9","Type":"ContainerStarted","Data":"a31cefce0af89989efd333596ac9d40239cc5d7b4e32abcefe76aed0b7d0ae73"} Jul 28 17:08:42.799597 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:42.799559 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" event={"ID":"42785eed679f9257ac8637d2df48ef77","Type":"ContainerStarted","Data":"0f125675bd2aefd2087593840bf2c230d87b24167d504dcd3a42c91477fb9b54"} Jul 28 17:08:42.817230 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:42.817159 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-215.ec2.internal" podStartSLOduration=3.817133776 podStartE2EDuration="3.817133776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:41.80458663 +0000 UTC m=+4.563447549" watchObservedRunningTime="2026-07-28 17:08:42.817133776 +0000 UTC m=+5.575994696" Jul 28 17:08:43.237150 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:43.237111 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:43.237363 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.237332 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:43.237432 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.237399 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret podName:5feed6d1-3fca-42ac-991a-4803dbe46b52 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:47.237380619 +0000 UTC m=+9.996241518 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret") pod "global-pull-secret-syncer-62dsj" (UID: "5feed6d1-3fca-42ac-991a-4803dbe46b52") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:43.339102 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:43.338430 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:43.339102 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.338607 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:43.339102 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.338679 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:47.338659784 +0000 UTC m=+10.097520694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:43.440410 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:43.439599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:43.440410 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.439870 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:43.440410 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.439892 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:43.440410 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.439923 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qlv9x for pod openshift-network-diagnostics/network-check-target-qtjtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:43.440410 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.440004 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x podName:93cfcf12-af34-48a1-81dd-4957742144ff nodeName:}" failed. No retries permitted until 2026-07-28 17:08:47.439983418 +0000 UTC m=+10.198844316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlv9x" (UniqueName: "kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x") pod "network-check-target-qtjtr" (UID: "93cfcf12-af34-48a1-81dd-4957742144ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:43.719112 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:43.718055 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:43.719112 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.718250 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:43.719112 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:43.718709 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:43.719112 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.718803 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:43.719112 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:43.718964 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:43.719112 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:43.719069 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:45.717380 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:45.717308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:45.717380 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:45.717329 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:45.717380 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:45.717308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:45.717966 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:45.717448 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:45.717966 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:45.717952 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:45.718071 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:45.718046 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:47.274809 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:47.274765 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:47.275314 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.274941 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:47.275314 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.275008 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret podName:5feed6d1-3fca-42ac-991a-4803dbe46b52 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:55.274990228 +0000 UTC m=+18.033851125 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret") pod "global-pull-secret-syncer-62dsj" (UID: "5feed6d1-3fca-42ac-991a-4803dbe46b52") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:47.375967 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:47.375927 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:47.376142 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.376112 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:47.376310 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.376178 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:08:55.376159444 +0000 UTC m=+18.135020354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:47.476835 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:47.476687 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:47.477028 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.476927 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:47.477028 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.476957 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:47.477028 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.476974 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qlv9x for pod openshift-network-diagnostics/network-check-target-qtjtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:47.477152 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.477040 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x podName:93cfcf12-af34-48a1-81dd-4957742144ff nodeName:}" failed. No retries permitted until 2026-07-28 17:08:55.477018102 +0000 UTC m=+18.235879013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlv9x" (UniqueName: "kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x") pod "network-check-target-qtjtr" (UID: "93cfcf12-af34-48a1-81dd-4957742144ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:47.718918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:47.718867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:47.719175 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:47.718972 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:47.719175 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:47.719005 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:47.719175 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.719012 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:47.719175 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.719088 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:47.719451 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:47.719173 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:49.718135 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:49.718100 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:49.718588 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:49.718233 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:49.718662 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:49.718631 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:49.718778 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:49.718735 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:49.718869 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:49.718790 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:49.718932 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:49.718909 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:51.717587 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:51.717556 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:51.718009 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:51.717597 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:51.718009 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:51.717674 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:51.718009 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:51.717779 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:51.718009 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:51.717906 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:51.718009 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:51.717998 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:53.718036 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:53.718001 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:53.718524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:53.718001 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:53.718524 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:53.718120 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:53.718524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:53.717998 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:53.718524 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:53.718196 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:53.718524 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:53.718268 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:55.336476 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:55.336439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:55.336814 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.336579 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:55.336814 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.336641 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret podName:5feed6d1-3fca-42ac-991a-4803dbe46b52 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:11.336623958 +0000 UTC m=+34.095484863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret") pod "global-pull-secret-syncer-62dsj" (UID: "5feed6d1-3fca-42ac-991a-4803dbe46b52") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:08:55.437781 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:55.437740 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:55.437947 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.437864 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:55.437947 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.437915 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:11.437901646 +0000 UTC m=+34.196762539 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:08:55.538570 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:55.538537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:55.538727 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.538696 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:08:55.538727 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.538714 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:08:55.538727 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.538723 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qlv9x for pod openshift-network-diagnostics/network-check-target-qtjtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:55.538843 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.538777 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x podName:93cfcf12-af34-48a1-81dd-4957742144ff nodeName:}" failed. No retries permitted until 2026-07-28 17:09:11.538761104 +0000 UTC m=+34.297622016 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlv9x" (UniqueName: "kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x") pod "network-check-target-qtjtr" (UID: "93cfcf12-af34-48a1-81dd-4957742144ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:08:55.721564 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:55.721532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:55.721752 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:55.721532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:55.721752 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.721656 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:55.721899 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.721757 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:55.721899 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:55.721532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:55.721899 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:55.721860 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:57.721140 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:57.721104 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:57.721639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:57.721146 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:57.721639 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:57.721222 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:57.721639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:57.721278 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:57.721639 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:57.721398 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:57.721639 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:57.721494 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:59.717497 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.717467 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:08:59.717891 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:59.717607 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:08:59.717891 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.717688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:08:59.717891 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:59.717771 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:08:59.717891 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.717856 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:08:59.718110 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:08:59.717926 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:08:59.836034 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.836004 2570 generic.go:356] "Generic (PLEG): container finished" podID="90d0c9d9-154c-4867-acd4-3c4a14069201" containerID="f48aef6564175c47077c2fd887dbb22cbeff3cbf5408274f1e4a9fc532340d7f" exitCode=0 Jul 28 17:08:59.836158 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.836080 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerDied","Data":"f48aef6564175c47077c2fd887dbb22cbeff3cbf5408274f1e4a9fc532340d7f"} Jul 28 17:08:59.838668 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.838647 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"c1400a2ac3d65a4096e612cba468caf4122422aed95bc881fffb04cf0de89109"} Jul 28 17:08:59.838763 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.838716 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"588c4175eca90f31e3434e8867c47e42f469a4cc0615712870449563181d6dac"} Jul 28 17:08:59.840231 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.840199 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-d92lh" event={"ID":"8e93e778-0d32-4bf2-8be0-7999c020ae7e","Type":"ContainerStarted","Data":"8c8dce8ae21eabe478a115f2149e8c5da2384f3ba0bcf10773e52f6e44dc2e52"} Jul 28 17:08:59.841572 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.841550 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vbj6k" event={"ID":"ba17a781-9e7c-4ec2-ab60-9a2d4132c464","Type":"ContainerStarted","Data":"96b7e8da7cbdad60d344b696bd9b080beaf35f62217b7e5825005af870b983a6"} Jul 28 17:08:59.843096 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.843077 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" event={"ID":"0638b35e-71d3-4b3f-b998-aea01f04aee1","Type":"ContainerStarted","Data":"951c34c7a756dd4e63927d8badc20bddfa7467b86f2580d6cc48996eb244fa5d"} Jul 28 17:08:59.844233 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.844214 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vdrdq" event={"ID":"48646f83-5284-4e24-8ea7-da85a74922aa","Type":"ContainerStarted","Data":"130ebe4df003e0e2603d2b06587f51b4e975fb9fd534cba4c8c9e52ba71cdfb5"} Jul 28 17:08:59.845633 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.845613 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t9m7d" event={"ID":"e569df22-dae2-4b3d-b50a-6fda9da8140b","Type":"ContainerStarted","Data":"89f87f5af163f333e47cfcd840f358157400efe9d7a4c7e7a9859864115c3c0e"} Jul 28 17:08:59.846900 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.846881 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-kcdsz" event={"ID":"2a9b8086-4250-4feb-baee-b561b9015044","Type":"ContainerStarted","Data":"d9e6f04bab72703eb6a9519397104fc24929dc572d76bc2d2a671cf15e408e4c"} Jul 28 17:08:59.862340 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.862281 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-215.ec2.internal" podStartSLOduration=20.862261453 podStartE2EDuration="20.862261453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:42.818278806 +0000 UTC m=+5.577139718" watchObservedRunningTime="2026-07-28 17:08:59.862261453 +0000 UTC m=+22.621122369" Jul 28 17:08:59.880084 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.880030 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-d92lh" podStartSLOduration=3.015465624 podStartE2EDuration="21.880013047s" totalImagesPullingTime="18.864547423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:59.87984961 +0000 UTC m=+22.638710518" watchObservedRunningTime="2026-07-28 17:08:59.880013047 +0000 UTC m=+22.638873964" Jul 28 17:08:59.902433 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.902386 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vdrdq" podStartSLOduration=3.063857472 podStartE2EDuration="21.902371165s" totalImagesPullingTime="18.838513693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:59.902197268 +0000 UTC m=+22.661058184" watchObservedRunningTime="2026-07-28 17:08:59.902371165 +0000 UTC m=+22.661232078" Jul 28 17:08:59.926791 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.926744 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vbj6k" podStartSLOduration=3.093925605 podStartE2EDuration="21.926730447s" totalImagesPullingTime="18.832804842s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:59.926169717 +0000 UTC m=+22.685030634" watchObservedRunningTime="2026-07-28 17:08:59.926730447 +0000 UTC m=+22.685591361" Jul 28 17:08:59.953035 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.952997 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-kcdsz" podStartSLOduration=7.336540303 podStartE2EDuration="21.952985616s" totalImagesPullingTime="14.616445313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:59.952896499 +0000 UTC m=+22.711757413" watchObservedRunningTime="2026-07-28 17:08:59.952985616 +0000 UTC m=+22.711846531" Jul 28 17:08:59.978641 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:08:59.977854 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t9m7d" podStartSLOduration=3.077611295 podStartE2EDuration="21.977821086s" totalImagesPullingTime="18.900209791s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:08:59.976969763 +0000 UTC m=+22.735830666" watchObservedRunningTime="2026-07-28 17:08:59.977821086 +0000 UTC m=+22.736682001" Jul 28 17:09:00.850975 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:00.850943 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tp7cx" event={"ID":"cc70882a-a450-4afd-9a42-8eaa3bcc61d6","Type":"ContainerStarted","Data":"445598693c6220bedb561dbf768b5505ee7b6f0fc0759ff48863305bd8db24be"} Jul 28 17:09:00.854238 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:00.854150 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"ae24c2089806d87b8911c9f56cfde59b221a4569b2e1e38385defcf565cc8568"} Jul 28 17:09:00.854360 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:00.854236 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"95e3de3d76e3133a7bfaf47521e32c5e22af669bf631d80e4a4d82f88d320b71"} Jul 28 17:09:00.854360 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:00.854293 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"aa8c92ca7949162eb710ce79a56371c846c3fd9b72dd8911a37ae3ea58370689"} Jul 28 17:09:00.854360 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:00.854346 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"78659fda80f0f0b076b96080a005e8cbfb3d9e5119d8faa2dd07a68a979f2300"} Jul 28 17:09:01.098795 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.098707 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 17:09:01.647002 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.646885 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:09:01.098730705Z","UUID":"566ffb74-279c-4216-b5b0-35c63bc7817a","Handler":null,"Name":"","Endpoint":""} Jul 28 17:09:01.648359 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.648335 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 17:09:01.648453 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.648376 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 17:09:01.718205 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.718175 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:01.718370 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.718175 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:01.718370 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:01.718276 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:09:01.718370 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.718175 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:01.718370 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:01.718357 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:09:01.718543 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:01.718452 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:09:01.859638 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.859566 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" event={"ID":"0638b35e-71d3-4b3f-b998-aea01f04aee1","Type":"ContainerStarted","Data":"df4f6b13af246a89a3ba538b0b529045680f02f680af05dc0c2db13350fa70ba"} Jul 28 17:09:01.876012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:01.875956 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tp7cx" podStartSLOduration=5.029709546 podStartE2EDuration="23.87593545s" totalImagesPullingTime="18.846225904s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:01.875377406 +0000 UTC m=+24.634238322" watchObservedRunningTime="2026-07-28 17:09:01.87593545 +0000 UTC m=+24.634796366" Jul 28 17:09:02.646551 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:02.646478 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:09:02.647157 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:02.647138 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:09:02.864447 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:02.864406 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" event={"ID":"0638b35e-71d3-4b3f-b998-aea01f04aee1","Type":"ContainerStarted","Data":"773419c77a41af69beaf2f82b0f47888e22d3078e3ac7e90d8df3ec1a161219c"} Jul 28 17:09:02.867882 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:02.867824 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"b99ecc32c1746827141643a2f73ef5b6334f75d2453b25c905c880c4f80c4791"} Jul 28 17:09:02.883626 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:02.883581 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9v8vg" podStartSLOduration=3.818251048 podStartE2EDuration="24.883568663s" totalImagesPullingTime="21.065317615s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:02.883482015 +0000 UTC m=+25.642342929" watchObservedRunningTime="2026-07-28 17:09:02.883568663 +0000 UTC m=+25.642429576" Jul 28 17:09:03.717428 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:03.717387 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:03.717428 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:03.717429 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:03.717671 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:03.717387 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:03.717671 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:03.717522 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:09:03.717671 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:03.717611 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:09:03.717855 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:03.717711 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:09:05.438341 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.438309 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:09:05.439078 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.438429 2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 28 17:09:05.439078 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.439018 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-kcdsz" Jul 28 17:09:05.717592 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.717516 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:05.717592 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.717536 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:05.717778 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.717517 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:05.717778 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:05.717622 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:09:05.717778 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:05.717686 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:09:05.717922 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:05.717779 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:09:05.878151 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.878115 2570 generic.go:356] "Generic (PLEG): container finished" podID="90d0c9d9-154c-4867-acd4-3c4a14069201" containerID="78c413fc5526fd0497234b9e3927779f7bc302bfd2b5c362f2e814c140e8190d" exitCode=0 Jul 28 17:09:05.878328 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.878199 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerDied","Data":"78c413fc5526fd0497234b9e3927779f7bc302bfd2b5c362f2e814c140e8190d"} Jul 28 17:09:05.881695 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.881673 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" event={"ID":"987609de-891a-47cb-b2da-262a39bda2fa","Type":"ContainerStarted","Data":"395fef06a7c3794e4edd0d1dff633028920a6be07f620f5db376868222d4a4f0"} Jul 28 17:09:05.882024 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.882005 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:09:05.897230 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.897207 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:09:05.941135 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:05.941088 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" podStartSLOduration=9.032498965 podStartE2EDuration="27.941073175s" totalImagesPullingTime="18.90857421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:05.939031155 +0000 UTC m=+28.697892064" watchObservedRunningTime="2026-07-28 17:09:05.941073175 +0000 UTC m=+28.699934088" Jul 28 17:09:06.652667 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:06.652418 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:09:06.884961 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:06.884934 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:09:06.899982 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:06.899897 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:09:07.091177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.091147 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-plwl2"] Jul 28 17:09:07.091334 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.091289 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:07.091413 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:07.091390 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:09:07.094249 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.094222 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-62dsj"] Jul 28 17:09:07.094373 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.094333 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:07.094447 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:07.094425 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:09:07.095008 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.094983 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtjtr"] Jul 28 17:09:07.095128 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.095076 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:07.095191 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:07.095167 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:09:07.888160 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.888122 2570 generic.go:356] "Generic (PLEG): container finished" podID="90d0c9d9-154c-4867-acd4-3c4a14069201" containerID="3e70ab700e946a88f6638f6680a70512e344ea419c87c6982018614dc2f18900" exitCode=0 Jul 28 17:09:07.888578 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:07.888203 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerDied","Data":"3e70ab700e946a88f6638f6680a70512e344ea419c87c6982018614dc2f18900"} Jul 28 17:09:08.717451 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:08.717416 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:08.717451 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:08.717448 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:08.717692 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:08.717448 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:08.717692 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:08.717530 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:09:08.717692 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:08.717590 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:09:08.717862 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:08.717701 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:09:09.894574 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:09.894534 2570 generic.go:356] "Generic (PLEG): container finished" podID="90d0c9d9-154c-4867-acd4-3c4a14069201" containerID="960d56772f9e3393d80bf89316d4211170318f18d765931ff5413eb3e2991d3e" exitCode=0 Jul 28 17:09:09.894951 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:09.894601 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerDied","Data":"960d56772f9e3393d80bf89316d4211170318f18d765931ff5413eb3e2991d3e"} Jul 28 17:09:10.718382 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:10.718139 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:10.718548 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:10.718186 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:10.718548 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:10.718489 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qtjtr" podUID="93cfcf12-af34-48a1-81dd-4957742144ff" Jul 28 17:09:10.718645 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:10.718575 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:09:10.718645 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:10.718189 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:10.718726 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:10.718685 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-62dsj" podUID="5feed6d1-3fca-42ac-991a-4803dbe46b52" Jul 28 17:09:11.355959 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:11.355927 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:11.356363 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.356084 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:11.356363 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.356150 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret podName:5feed6d1-3fca-42ac-991a-4803dbe46b52 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:43.356132885 +0000 UTC m=+66.114993795 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret") pod "global-pull-secret-syncer-62dsj" (UID: "5feed6d1-3fca-42ac-991a-4803dbe46b52") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:09:11.457120 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:11.457083 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:11.457265 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.457220 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:11.457321 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.457291 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:43.457276297 +0000 UTC m=+66.216137188 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:09:11.558214 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:11.558170 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:11.558389 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.558348 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:09:11.558389 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.558376 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:09:11.558389 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.558390 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qlv9x for pod openshift-network-diagnostics/network-check-target-qtjtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:11.558551 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:11.558459 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x podName:93cfcf12-af34-48a1-81dd-4957742144ff nodeName:}" failed. No retries permitted until 2026-07-28 17:09:43.558438611 +0000 UTC m=+66.317299509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qlv9x" (UniqueName: "kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x") pod "network-check-target-qtjtr" (UID: "93cfcf12-af34-48a1-81dd-4957742144ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:09:12.579933 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.579903 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-215.ec2.internal" event="NodeReady" Jul 28 17:09:12.580328 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.580040 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 17:09:12.628701 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.628669 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-587668d4b-dp65p"] Jul 28 17:09:12.631290 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.631269 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.635374 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.635344 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 17:09:12.635503 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.635464 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-brvwl\"" Jul 28 17:09:12.635665 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.635649 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 17:09:12.636959 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.636936 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 17:09:12.641925 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.641902 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 17:09:12.646448 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.646223 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mjf49"] Jul 28 17:09:12.648322 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.648285 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-587668d4b-dp65p"] Jul 28 17:09:12.648432 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.648394 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.651303 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.651281 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwxn2\"" Jul 28 17:09:12.651794 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.651625 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 17:09:12.651794 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.651733 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 17:09:12.651978 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.651965 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r2rw6"] Jul 28 17:09:12.653905 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.653886 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:12.658067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.658045 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 17:09:12.658190 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.658142 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 17:09:12.658264 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.658247 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jfwmx\"" Jul 28 17:09:12.658636 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.658614 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 17:09:12.668185 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.668166 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mjf49"] Jul 28 17:09:12.673877 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.673855 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r2rw6"] Jul 28 17:09:12.717946 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.717913 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:12.717946 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.717949 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:12.718172 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.717953 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:12.720857 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.720821 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:09:12.721415 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.721159 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:09:12.721415 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.721167 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:09:12.721415 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.721252 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-99hnw\"" Jul 28 17:09:12.722098 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.721995 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8h5hq\"" Jul 28 17:09:12.722098 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.722062 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:09:12.766544 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjjvz\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-kube-api-access-tjjvz\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.766730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766555 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36520022-e419-412e-bf8b-d24c5da77c7d-tmp-dir\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.766730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766583 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-certificates\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.766730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766656 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sfjh\" (UniqueName: \"kubernetes.io/projected/36520022-e419-412e-bf8b-d24c5da77c7d-kube-api-access-2sfjh\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.766730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766711 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36520022-e419-412e-bf8b-d24c5da77c7d-config-volume\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.766950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-installation-pull-secrets\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.766950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.766950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-bound-sa-token\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.766950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766852 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00ba8b0-d3ea-4a1c-8967-6d48517b3568-cert\") pod \"ingress-canary-r2rw6\" (UID: \"c00ba8b0-d3ea-4a1c-8967-6d48517b3568\") " pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:12.766950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766882 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.766950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766929 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-trusted-ca\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.766950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766948 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp6cr\" (UniqueName: \"kubernetes.io/projected/c00ba8b0-d3ea-4a1c-8967-6d48517b3568-kube-api-access-wp6cr\") pod \"ingress-canary-r2rw6\" (UID: \"c00ba8b0-d3ea-4a1c-8967-6d48517b3568\") " pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:12.767234 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.766975 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf01f69-9d9b-4bc5-ba76-47c2fe585884-ca-trust-extracted\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.767234 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.767000 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-image-registry-private-configuration\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.867860 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.867811 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf01f69-9d9b-4bc5-ba76-47c2fe585884-ca-trust-extracted\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868028 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.867954 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-image-registry-private-configuration\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868028 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.867998 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjjvz\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-kube-api-access-tjjvz\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868154 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36520022-e419-412e-bf8b-d24c5da77c7d-tmp-dir\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.868154 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868072 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-certificates\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868154 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868129 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sfjh\" (UniqueName: \"kubernetes.io/projected/36520022-e419-412e-bf8b-d24c5da77c7d-kube-api-access-2sfjh\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.868306 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868187 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36520022-e419-412e-bf8b-d24c5da77c7d-config-volume\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.868306 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-installation-pull-secrets\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868306 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868259 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.868306 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868291 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf01f69-9d9b-4bc5-ba76-47c2fe585884-ca-trust-extracted\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868298 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-bound-sa-token\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868357 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36520022-e419-412e-bf8b-d24c5da77c7d-tmp-dir\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.868496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868360 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00ba8b0-d3ea-4a1c-8967-6d48517b3568-cert\") pod \"ingress-canary-r2rw6\" (UID: \"c00ba8b0-d3ea-4a1c-8967-6d48517b3568\") " pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:12.868496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868465 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-trusted-ca\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.868496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868489 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp6cr\" (UniqueName: \"kubernetes.io/projected/c00ba8b0-d3ea-4a1c-8967-6d48517b3568-kube-api-access-wp6cr\") pod \"ingress-canary-r2rw6\" (UID: \"c00ba8b0-d3ea-4a1c-8967-6d48517b3568\") " pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:12.868812 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:12.868793 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:12.868812 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:12.868814 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:09:12.868972 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:12.868886 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:13.368869121 +0000 UTC m=+36.127730014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:09:12.868972 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868884 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36520022-e419-412e-bf8b-d24c5da77c7d-config-volume\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.868972 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.868892 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-certificates\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.869126 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:12.868982 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:12.869126 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:12.869031 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:09:13.36901395 +0000 UTC m=+36.127874844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:09:12.869774 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.869722 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-trusted-ca\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.872747 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.872723 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-installation-pull-secrets\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.872881 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.872723 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-image-registry-private-configuration\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.872881 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.872812 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c00ba8b0-d3ea-4a1c-8967-6d48517b3568-cert\") pod \"ingress-canary-r2rw6\" (UID: \"c00ba8b0-d3ea-4a1c-8967-6d48517b3568\") " pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:12.880100 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.880074 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sfjh\" (UniqueName: \"kubernetes.io/projected/36520022-e419-412e-bf8b-d24c5da77c7d-kube-api-access-2sfjh\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:12.881311 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.881290 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-bound-sa-token\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.881617 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.881595 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjjvz\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-kube-api-access-tjjvz\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:12.882246 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.882224 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp6cr\" (UniqueName: \"kubernetes.io/projected/c00ba8b0-d3ea-4a1c-8967-6d48517b3568-kube-api-access-wp6cr\") pod \"ingress-canary-r2rw6\" (UID: \"c00ba8b0-d3ea-4a1c-8967-6d48517b3568\") " pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:12.907717 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.907675 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:12.965884 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:12.965850 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r2rw6" Jul 28 17:09:13.030084 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:13.030045 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:13.373469 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:13.373429 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:13.373652 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:13.373554 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:13.373652 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:13.373581 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:13.373652 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:13.373605 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:09:13.373791 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:13.373666 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:14.373645402 +0000 UTC m=+37.132506317 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:09:13.373791 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:13.373668 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:13.373791 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:13.373718 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:09:14.373700825 +0000 UTC m=+37.132561724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:09:13.801624 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:13.801590 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r2rw6"] Jul 28 17:09:13.805529 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:09:13.805505 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00ba8b0_d3ea_4a1c_8967_6d48517b3568.slice/crio-998c9764fbdaba901a291e2a89a95d8511ae5597c48c50a38799c0f761431782 WatchSource:0}: Error finding container 998c9764fbdaba901a291e2a89a95d8511ae5597c48c50a38799c0f761431782: Status 404 returned error can't find the container with id 998c9764fbdaba901a291e2a89a95d8511ae5597c48c50a38799c0f761431782 Jul 28 17:09:13.906662 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:13.906445 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r2rw6" event={"ID":"c00ba8b0-d3ea-4a1c-8967-6d48517b3568","Type":"ContainerStarted","Data":"998c9764fbdaba901a291e2a89a95d8511ae5597c48c50a38799c0f761431782"} Jul 28 17:09:13.909429 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:13.909403 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerStarted","Data":"caed37ef5f0c71bbd914e0ad15a4518b66ee8b381b375d9bd842e236c0971be7"} Jul 28 17:09:14.382018 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:14.381974 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:14.382190 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:14.382094 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:14.382190 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:14.382147 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:14.382190 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:14.382168 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:09:14.382356 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:14.382212 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:14.382356 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:14.382238 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:16.382217204 +0000 UTC m=+39.141078098 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:09:14.382356 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:14.382263 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:09:16.382247286 +0000 UTC m=+39.141108178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:09:14.915813 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:14.915780 2570 generic.go:356] "Generic (PLEG): container finished" podID="90d0c9d9-154c-4867-acd4-3c4a14069201" containerID="caed37ef5f0c71bbd914e0ad15a4518b66ee8b381b375d9bd842e236c0971be7" exitCode=0 Jul 28 17:09:14.916247 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:14.915868 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerDied","Data":"caed37ef5f0c71bbd914e0ad15a4518b66ee8b381b375d9bd842e236c0971be7"} Jul 28 17:09:15.922507 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:15.922470 2570 generic.go:356] "Generic (PLEG): container finished" podID="90d0c9d9-154c-4867-acd4-3c4a14069201" containerID="5f3461557ff77bb737f92312f337f868604f4214d473eeb7e3c8e10297665bca" exitCode=0 Jul 28 17:09:15.922995 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:15.922533 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerDied","Data":"5f3461557ff77bb737f92312f337f868604f4214d473eeb7e3c8e10297665bca"} Jul 28 17:09:16.397474 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:16.397450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:16.397557 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:16.397539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:16.397621 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:16.397604 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:16.397657 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:16.397624 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:16.397687 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:16.397670 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:09:20.397657685 +0000 UTC m=+43.156518577 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:09:16.397734 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:16.397626 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:09:16.397767 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:16.397738 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:20.397726653 +0000 UTC m=+43.156587565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:09:16.926697 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:16.926658 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r2rw6" event={"ID":"c00ba8b0-d3ea-4a1c-8967-6d48517b3568","Type":"ContainerStarted","Data":"90eef836c372a5fda4b2c8a721998fb9288f8201893fec27b09b02370dc263a7"} Jul 28 17:09:16.929933 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:16.929903 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" event={"ID":"90d0c9d9-154c-4867-acd4-3c4a14069201","Type":"ContainerStarted","Data":"33d7f4db3ddd0ded6504d0651cff2cdeb85a762558c8ad8c6885ceb3641f2876"} Jul 28 17:09:16.952322 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:16.952270 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r2rw6" podStartSLOduration=2.401972121 podStartE2EDuration="4.952254357s" totalImagesPullingTime="2.550282236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:16.951752868 +0000 UTC m=+39.710613783" watchObservedRunningTime="2026-07-28 17:09:16.952254357 +0000 UTC m=+39.711115272" Jul 28 17:09:16.975769 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:16.975719 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c9hbp" podStartSLOduration=8.912098675 podStartE2EDuration="38.975702046s" totalImagesPullingTime="30.063603371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:16.975122363 +0000 UTC m=+39.733983279" watchObservedRunningTime="2026-07-28 17:09:16.975702046 +0000 UTC m=+39.734562961" Jul 28 17:09:17.114314 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:17.114279 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:17.216237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:17.216146 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:17.766474 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:17.766292 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:17.867696 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:17.867654 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:20.429143 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:20.429096 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:20.429531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:20.429207 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:20.429531 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:20.429283 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:20.429531 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:20.429298 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:20.429531 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:20.429302 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:09:20.429531 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:20.429365 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.429345674 +0000 UTC m=+51.188206581 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:09:20.429531 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:20.429378 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:09:28.429372558 +0000 UTC m=+51.188233450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:09:21.052368 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:21.052334 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:21.146393 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:21.146360 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:23.054286 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:23.054244 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:23.125647 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:23.125617 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:23.643580 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:23.643550 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:23.744320 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:23.744279 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" limit=1024 Jul 28 17:09:28.485054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:28.485017 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:28.485439 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:28.485163 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:28.485439 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:28.485197 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:28.485439 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:28.485221 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:09:28.485439 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:28.485260 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:28.485439 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:28.485290 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:09:44.485273542 +0000 UTC m=+67.244134433 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:09:28.485439 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:28.485303 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:09:44.485297004 +0000 UTC m=+67.244157896 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:09:38.900481 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:38.900444 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r9ndn" Jul 28 17:09:43.393321 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.393284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:43.396638 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.396615 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:09:43.408054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.408023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5feed6d1-3fca-42ac-991a-4803dbe46b52-original-pull-secret\") pod \"global-pull-secret-syncer-62dsj\" (UID: \"5feed6d1-3fca-42ac-991a-4803dbe46b52\") " pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:43.494396 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.494357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:09:43.497462 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.497444 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:09:43.505392 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:43.505369 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 17:09:43.505469 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:43.505452 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:47.50543455 +0000 UTC m=+130.264295442 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : secret "metrics-daemon-secret" not found Jul 28 17:09:43.595679 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.595645 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:43.598928 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.598910 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:09:43.609323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.609293 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:09:43.619203 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.619168 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlv9x\" (UniqueName: \"kubernetes.io/projected/93cfcf12-af34-48a1-81dd-4957742144ff-kube-api-access-qlv9x\") pod \"network-check-target-qtjtr\" (UID: \"93cfcf12-af34-48a1-81dd-4957742144ff\") " pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:43.629850 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.629813 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-62dsj" Jul 28 17:09:43.638062 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.638040 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8h5hq\"" Jul 28 17:09:43.645689 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.645616 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:43.768786 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.768733 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-62dsj"] Jul 28 17:09:43.771822 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:09:43.771794 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5feed6d1_3fca_42ac_991a_4803dbe46b52.slice/crio-447817cd9ed556a577a10085dfcaa39bd4216ef9ab4064b577349f2a5b772413 WatchSource:0}: Error finding container 447817cd9ed556a577a10085dfcaa39bd4216ef9ab4064b577349f2a5b772413: Status 404 returned error can't find the container with id 447817cd9ed556a577a10085dfcaa39bd4216ef9ab4064b577349f2a5b772413 Jul 28 17:09:43.797612 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.797576 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qtjtr"] Jul 28 17:09:43.801372 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:09:43.801346 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93cfcf12_af34_48a1_81dd_4957742144ff.slice/crio-6c5e24cf8de84c1c19fb8b04268ea024386429996d78751a8fe77b5a62b4b99b WatchSource:0}: Error finding container 6c5e24cf8de84c1c19fb8b04268ea024386429996d78751a8fe77b5a62b4b99b: Status 404 returned error can't find the container with id 6c5e24cf8de84c1c19fb8b04268ea024386429996d78751a8fe77b5a62b4b99b Jul 28 17:09:43.989890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.989778 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtjtr" event={"ID":"93cfcf12-af34-48a1-81dd-4957742144ff","Type":"ContainerStarted","Data":"6c5e24cf8de84c1c19fb8b04268ea024386429996d78751a8fe77b5a62b4b99b"} Jul 28 17:09:43.990783 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:43.990759 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-62dsj" event={"ID":"5feed6d1-3fca-42ac-991a-4803dbe46b52","Type":"ContainerStarted","Data":"447817cd9ed556a577a10085dfcaa39bd4216ef9ab4064b577349f2a5b772413"} Jul 28 17:09:44.505177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:44.505137 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:09:44.505688 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:44.505207 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:09:44.505688 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:44.505328 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:09:44.505688 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:44.505419 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:10:16.505399078 +0000 UTC m=+99.264259974 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:09:44.505688 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:44.505331 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:09:44.505688 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:44.505456 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:09:44.505688 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:09:44.505504 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:16.505493187 +0000 UTC m=+99.264354096 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:09:49.003951 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:49.003917 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qtjtr" event={"ID":"93cfcf12-af34-48a1-81dd-4957742144ff","Type":"ContainerStarted","Data":"b07e2b668106e1d36fa745d405538e80ac591b2a9d9b5071b73bbb9a70d47749"} Jul 28 17:09:49.004382 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:49.003993 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:09:49.005336 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:49.005315 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-62dsj" event={"ID":"5feed6d1-3fca-42ac-991a-4803dbe46b52","Type":"ContainerStarted","Data":"d2c465f12a45abf68ef9efcebd94f677aefed366839f0d3f66892f10928adcbe"} Jul 28 17:09:49.023291 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:49.023244 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qtjtr" podStartSLOduration=66.592386638 podStartE2EDuration="1m11.023230804s" totalImagesPullingTime="4.430844166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:49.02251186 +0000 UTC m=+71.781372773" watchObservedRunningTime="2026-07-28 17:09:49.023230804 +0000 UTC m=+71.782091696" Jul 28 17:09:49.039736 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:09:49.039683 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-62dsj" podStartSLOduration=66.576260007 podStartE2EDuration="1m11.039668823s" totalImagesPullingTime="4.463408816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:09:49.039032905 +0000 UTC m=+71.797893818" watchObservedRunningTime="2026-07-28 17:09:49.039668823 +0000 UTC m=+71.798529789" Jul 28 17:10:16.524204 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:10:16.524076 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:10:16.524204 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:10:16.524130 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:10:16.524743 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:10:16.524232 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:16.524743 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:10:16.524243 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:10:16.524743 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:10:16.524250 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:16.524743 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:10:16.524305 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:20.524288284 +0000 UTC m=+163.283149176 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:10:16.524743 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:10:16.524346 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:11:20.524330796 +0000 UTC m=+163.283191688 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:10:20.009866 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:10:20.009813 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qtjtr" Jul 28 17:10:47.536451 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:10:47.536383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:10:47.536950 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:10:47.536534 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 17:10:47.536950 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:10:47.536595 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:49.536577465 +0000 UTC m=+252.295438375 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : secret "metrics-daemon-secret" not found Jul 28 17:11:15.645613 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:15.645572 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-587668d4b-dp65p" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" Jul 28 17:11:15.660751 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:15.660718 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mjf49" podUID="36520022-e419-412e-bf8b-d24c5da77c7d" Jul 28 17:11:15.741031 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:15.740982 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:11:16.207510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:11:16.207482 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:11:16.207510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:11:16.207510 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjf49" Jul 28 17:11:20.573506 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:11:20.573459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:11:20.573919 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:20.573609 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:11:20.573919 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:11:20.573619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:11:20.573919 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:20.573628 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-587668d4b-dp65p: secret "image-registry-tls" not found Jul 28 17:11:20.573919 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:20.573670 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:11:20.573919 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:20.573716 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls podName:baf01f69-9d9b-4bc5-ba76-47c2fe585884 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:22.573698336 +0000 UTC m=+285.332559245 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls") pod "image-registry-587668d4b-dp65p" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884") : secret "image-registry-tls" not found Jul 28 17:11:20.573919 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:11:20.573730 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls podName:36520022-e419-412e-bf8b-d24c5da77c7d nodeName:}" failed. No retries permitted until 2026-07-28 17:13:22.573724374 +0000 UTC m=+285.332585266 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls") pod "dns-default-mjf49" (UID: "36520022-e419-412e-bf8b-d24c5da77c7d") : secret "dns-default-metrics-tls" not found Jul 28 17:11:29.717930 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:11:29.717889 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:12:49.588723 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:49.588678 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:12:49.589192 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:49.588786 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 28 17:12:49.589192 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:49.588862 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs podName:b00417f8-bed7-4fcb-b3ec-f912fc8dfb91 nodeName:}" failed. No retries permitted until 2026-07-28 17:14:51.588845032 +0000 UTC m=+374.347705929 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs") pod "network-metrics-daemon-plwl2" (UID: "b00417f8-bed7-4fcb-b3ec-f912fc8dfb91") : secret "metrics-daemon-secret" not found Jul 28 17:12:53.143458 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.143423 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9522p"] Jul 28 17:12:53.145097 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.145080 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:53.147799 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.147778 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 28 17:12:53.147912 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.147878 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-rmdgr\"" Jul 28 17:12:53.148024 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.148009 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 28 17:12:53.150381 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.150359 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-z6qvn"] Jul 28 17:12:53.152081 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.152062 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6"] Jul 28 17:12:53.152196 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.152181 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.153665 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.153646 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t"] Jul 28 17:12:53.153777 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.153761 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.155175 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.155150 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 17:12:53.155175 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.155160 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" Jul 28 17:12:53.155325 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.155216 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 17:12:53.155325 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.155231 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 28 17:12:53.155510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.155496 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 28 17:12:53.155564 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.155518 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-pt64g\"" Jul 28 17:12:53.156236 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.156217 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 17:12:53.157193 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.157177 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 17:12:53.157593 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.157572 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 17:12:53.157673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.157594 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-sszp7\"" Jul 28 17:12:53.158264 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.158245 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 17:12:53.158366 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.158293 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bd2bv\"" Jul 28 17:12:53.161617 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.161598 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9522p"] Jul 28 17:12:53.163859 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.163818 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 28 17:12:53.166019 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.165993 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t"] Jul 28 17:12:53.169293 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.169275 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6"] Jul 28 17:12:53.175883 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.175866 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-z6qvn"] Jul 28 17:12:53.210422 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5530c61-043a-404e-afad-43651d677628-service-ca-bundle\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.210599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210447 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc96j\" (UniqueName: \"kubernetes.io/projected/a5530c61-043a-404e-afad-43651d677628-kube-api-access-sc96j\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.210599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210467 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:53.210599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210511 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a5530c61-043a-404e-afad-43651d677628-snapshots\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.210599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210540 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5530c61-043a-404e-afad-43651d677628-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.210599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlnwd\" (UniqueName: \"kubernetes.io/projected/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-kube-api-access-hlnwd\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.210599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210582 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqjtm\" (UniqueName: \"kubernetes.io/projected/358e9a0a-b26b-4b6a-b53c-2d83fea72f48-kube-api-access-gqjtm\") pod \"network-check-source-7d88cb55d8-q247t\" (UID: \"358e9a0a-b26b-4b6a-b53c-2d83fea72f48\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" Jul 28 17:12:53.210599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210601 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.210858 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210678 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5530c61-043a-404e-afad-43651d677628-tmp\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.210858 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210704 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5530c61-043a-404e-afad-43651d677628-serving-cert\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.210858 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.210858 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.210744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b79145a-1a31-4970-82af-1212de987dc9-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:53.240260 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.240227 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7"] Jul 28 17:12:53.242707 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.242688 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh"] Jul 28 17:12:53.242839 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.242815 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" Jul 28 17:12:53.244496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.244476 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.245744 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.245725 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 28 17:12:53.245850 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.245752 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:12:53.246040 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.246026 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-98pfh\"" Jul 28 17:12:53.247458 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.247443 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:12:53.247540 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.247517 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 28 17:12:53.247840 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.247816 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-8mr25\"" Jul 28 17:12:53.248303 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.248290 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 28 17:12:53.248678 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.248662 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 28 17:12:53.257951 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.257926 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh"] Jul 28 17:12:53.258788 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.258765 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7"] Jul 28 17:12:53.311902 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.311870 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.312067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.311917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5530c61-043a-404e-afad-43651d677628-tmp\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.312067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.311940 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1525dd4-496e-4b33-964e-db81ff558ed9-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.312067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.311964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1525dd4-496e-4b33-964e-db81ff558ed9-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.312533 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.312501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv8xv\" (UniqueName: \"kubernetes.io/projected/1d65527e-a4a8-465f-81c0-2bd8b441d91b-kube-api-access-rv8xv\") pod \"volume-data-source-validator-549c6d54cf-qscf7\" (UID: \"1d65527e-a4a8-465f-81c0-2bd8b441d91b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" Jul 28 17:12:53.312709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.312691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a5530c61-043a-404e-afad-43651d677628-snapshots\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.312814 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.312800 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5530c61-043a-404e-afad-43651d677628-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.312960 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.312942 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc96j\" (UniqueName: \"kubernetes.io/projected/a5530c61-043a-404e-afad-43651d677628-kube-api-access-sc96j\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.313060 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.313037 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.313118 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.313046 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlnwd\" (UniqueName: \"kubernetes.io/projected/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-kube-api-access-hlnwd\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.313118 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.312547 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5530c61-043a-404e-afad-43651d677628-tmp\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.313593 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.313570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqjtm\" (UniqueName: \"kubernetes.io/projected/358e9a0a-b26b-4b6a-b53c-2d83fea72f48-kube-api-access-gqjtm\") pod \"network-check-source-7d88cb55d8-q247t\" (UID: \"358e9a0a-b26b-4b6a-b53c-2d83fea72f48\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" Jul 28 17:12:53.313823 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.313807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5530c61-043a-404e-afad-43651d677628-serving-cert\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.314023 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.314007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.314131 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.314118 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b79145a-1a31-4970-82af-1212de987dc9-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:53.314226 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.314214 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5530c61-043a-404e-afad-43651d677628-service-ca-bundle\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.314330 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.314317 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:53.314428 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.314415 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj2x8\" (UniqueName: \"kubernetes.io/projected/b1525dd4-496e-4b33-964e-db81ff558ed9-kube-api-access-kj2x8\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.315131 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.315110 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/a5530c61-043a-404e-afad-43651d677628-snapshots\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.316153 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.316132 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:53.316266 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.316205 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls podName:cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:53.816189999 +0000 UTC m=+256.575050891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-thnp6" (UID: "cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:53.316336 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.316317 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:12:53.316395 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.316355 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert podName:3b79145a-1a31-4970-82af-1212de987dc9 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:53.816346484 +0000 UTC m=+256.575207375 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9522p" (UID: "3b79145a-1a31-4970-82af-1212de987dc9") : secret "networking-console-plugin-cert" not found Jul 28 17:12:53.317007 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.316979 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5530c61-043a-404e-afad-43651d677628-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.317192 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.317169 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5530c61-043a-404e-afad-43651d677628-service-ca-bundle\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.317379 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.317358 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b79145a-1a31-4970-82af-1212de987dc9-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:53.318918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.318899 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a5530c61-043a-404e-afad-43651d677628-serving-cert\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.321974 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.321957 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlnwd\" (UniqueName: \"kubernetes.io/projected/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-kube-api-access-hlnwd\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.324981 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.324960 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqjtm\" (UniqueName: \"kubernetes.io/projected/358e9a0a-b26b-4b6a-b53c-2d83fea72f48-kube-api-access-gqjtm\") pod \"network-check-source-7d88cb55d8-q247t\" (UID: \"358e9a0a-b26b-4b6a-b53c-2d83fea72f48\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" Jul 28 17:12:53.327890 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.327869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc96j\" (UniqueName: \"kubernetes.io/projected/a5530c61-043a-404e-afad-43651d677628-kube-api-access-sc96j\") pod \"insights-operator-5885fdb759-z6qvn\" (UID: \"a5530c61-043a-404e-afad-43651d677628\") " pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.338909 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.338886 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz"] Jul 28 17:12:53.341451 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.341434 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.342172 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.342150 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw"] Jul 28 17:12:53.344208 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.344188 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:53.344565 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.344546 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 28 17:12:53.344763 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.344742 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 28 17:12:53.344763 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.344753 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qwhkn\"" Jul 28 17:12:53.344911 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.344823 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:12:53.345106 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.345093 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 28 17:12:53.346772 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.346756 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:12:53.346861 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.346824 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 28 17:12:53.346975 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.346959 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-sl8r8\"" Jul 28 17:12:53.348659 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.348637 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 28 17:12:53.352619 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.352602 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz"] Jul 28 17:12:53.369237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.369216 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw"] Jul 28 17:12:53.415193 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415098 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:53.415193 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415182 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46326b21-01e9-4e63-85c8-516ea6d5d862-serving-cert\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.415374 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415209 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj2x8\" (UniqueName: \"kubernetes.io/projected/b1525dd4-496e-4b33-964e-db81ff558ed9-kube-api-access-kj2x8\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.415374 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46326b21-01e9-4e63-85c8-516ea6d5d862-config\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.415374 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415261 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qbx7\" (UniqueName: \"kubernetes.io/projected/46326b21-01e9-4e63-85c8-516ea6d5d862-kube-api-access-4qbx7\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.415535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1525dd4-496e-4b33-964e-db81ff558ed9-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.415535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415408 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1525dd4-496e-4b33-964e-db81ff558ed9-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.415535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415431 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnqgq\" (UniqueName: \"kubernetes.io/projected/10afb4aa-18a9-4ef8-ab02-295de0442376-kube-api-access-xnqgq\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:53.415535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415479 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv8xv\" (UniqueName: \"kubernetes.io/projected/1d65527e-a4a8-465f-81c0-2bd8b441d91b-kube-api-access-rv8xv\") pod \"volume-data-source-validator-549c6d54cf-qscf7\" (UID: \"1d65527e-a4a8-465f-81c0-2bd8b441d91b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" Jul 28 17:12:53.415980 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.415963 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1525dd4-496e-4b33-964e-db81ff558ed9-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.417554 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.417534 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1525dd4-496e-4b33-964e-db81ff558ed9-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.426299 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.426276 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv8xv\" (UniqueName: \"kubernetes.io/projected/1d65527e-a4a8-465f-81c0-2bd8b441d91b-kube-api-access-rv8xv\") pod \"volume-data-source-validator-549c6d54cf-qscf7\" (UID: \"1d65527e-a4a8-465f-81c0-2bd8b441d91b\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" Jul 28 17:12:53.427053 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.427036 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj2x8\" (UniqueName: \"kubernetes.io/projected/b1525dd4-496e-4b33-964e-db81ff558ed9-kube-api-access-kj2x8\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh\" (UID: \"b1525dd4-496e-4b33-964e-db81ff558ed9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.464143 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.464115 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-z6qvn" Jul 28 17:12:53.476609 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.476586 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" Jul 28 17:12:53.515949 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.515920 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:53.516092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.515982 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46326b21-01e9-4e63-85c8-516ea6d5d862-serving-cert\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.516092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.516001 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46326b21-01e9-4e63-85c8-516ea6d5d862-config\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.516092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.516023 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qbx7\" (UniqueName: \"kubernetes.io/projected/46326b21-01e9-4e63-85c8-516ea6d5d862-kube-api-access-4qbx7\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.516092 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.516055 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnqgq\" (UniqueName: \"kubernetes.io/projected/10afb4aa-18a9-4ef8-ab02-295de0442376-kube-api-access-xnqgq\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:53.516437 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.516419 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:12:53.516477 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.516471 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls podName:10afb4aa-18a9-4ef8-ab02-295de0442376 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:54.01645796 +0000 UTC m=+256.775318852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls") pod "cluster-samples-operator-6748489656-hf9bw" (UID: "10afb4aa-18a9-4ef8-ab02-295de0442376") : secret "samples-operator-tls" not found Jul 28 17:12:53.517433 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.517412 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/46326b21-01e9-4e63-85c8-516ea6d5d862-config\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.518756 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.518717 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/46326b21-01e9-4e63-85c8-516ea6d5d862-serving-cert\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.527238 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.527177 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qbx7\" (UniqueName: \"kubernetes.io/projected/46326b21-01e9-4e63-85c8-516ea6d5d862-kube-api-access-4qbx7\") pod \"service-ca-operator-5d6c75f84f-ddhsz\" (UID: \"46326b21-01e9-4e63-85c8-516ea6d5d862\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.528033 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.527981 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnqgq\" (UniqueName: \"kubernetes.io/projected/10afb4aa-18a9-4ef8-ab02-295de0442376-kube-api-access-xnqgq\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:53.553200 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.553174 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" Jul 28 17:12:53.559356 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.559331 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" Jul 28 17:12:53.599076 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.599015 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-z6qvn"] Jul 28 17:12:53.602746 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:12:53.602710 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5530c61_043a_404e_afad_43651d677628.slice/crio-078b50b90f56fa5cbccfcede693bce0c85cd8a9f00f67f4a355f15e1f46de8b8 WatchSource:0}: Error finding container 078b50b90f56fa5cbccfcede693bce0c85cd8a9f00f67f4a355f15e1f46de8b8: Status 404 returned error can't find the container with id 078b50b90f56fa5cbccfcede693bce0c85cd8a9f00f67f4a355f15e1f46de8b8 Jul 28 17:12:53.612335 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.612308 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t"] Jul 28 17:12:53.615522 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:12:53.615494 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod358e9a0a_b26b_4b6a_b53c_2d83fea72f48.slice/crio-4b65cc6f0684594846319da270ee765ec5f79098d6e9efc474b52dca07cb8dcb WatchSource:0}: Error finding container 4b65cc6f0684594846319da270ee765ec5f79098d6e9efc474b52dca07cb8dcb: Status 404 returned error can't find the container with id 4b65cc6f0684594846319da270ee765ec5f79098d6e9efc474b52dca07cb8dcb Jul 28 17:12:53.651614 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.651587 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" Jul 28 17:12:53.709787 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.709762 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh"] Jul 28 17:12:53.712653 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:12:53.712609 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1525dd4_496e_4b33_964e_db81ff558ed9.slice/crio-6ab73362f56e88bce0709eb2101bce1dcf9d6573837daa4f7a86b039cf5c8e8a WatchSource:0}: Error finding container 6ab73362f56e88bce0709eb2101bce1dcf9d6573837daa4f7a86b039cf5c8e8a: Status 404 returned error can't find the container with id 6ab73362f56e88bce0709eb2101bce1dcf9d6573837daa4f7a86b039cf5c8e8a Jul 28 17:12:53.725297 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.725234 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7"] Jul 28 17:12:53.729962 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:12:53.729939 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d65527e_a4a8_465f_81c0_2bd8b441d91b.slice/crio-aca24e1b1093c2394a9b20258d78b85c98115efa7b782a8121ae3d4356c4ef64 WatchSource:0}: Error finding container aca24e1b1093c2394a9b20258d78b85c98115efa7b782a8121ae3d4356c4ef64: Status 404 returned error can't find the container with id aca24e1b1093c2394a9b20258d78b85c98115efa7b782a8121ae3d4356c4ef64 Jul 28 17:12:53.795148 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.795081 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz"] Jul 28 17:12:53.797452 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:12:53.797427 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46326b21_01e9_4e63_85c8_516ea6d5d862.slice/crio-be12ca8e8037efddcf3330961d1c27bb20b91c6a5c5b6daaa68ac38401b76b50 WatchSource:0}: Error finding container be12ca8e8037efddcf3330961d1c27bb20b91c6a5c5b6daaa68ac38401b76b50: Status 404 returned error can't find the container with id be12ca8e8037efddcf3330961d1c27bb20b91c6a5c5b6daaa68ac38401b76b50 Jul 28 17:12:53.818142 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.818105 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:53.818238 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:53.818161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:53.818304 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.818238 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:53.818304 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.818297 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls podName:cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:54.818283246 +0000 UTC m=+257.577144139 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-thnp6" (UID: "cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:53.818407 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.818317 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:12:53.818407 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:53.818374 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert podName:3b79145a-1a31-4970-82af-1212de987dc9 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:54.818355254 +0000 UTC m=+257.577216160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9522p" (UID: "3b79145a-1a31-4970-82af-1212de987dc9") : secret "networking-console-plugin-cert" not found Jul 28 17:12:54.020276 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.020170 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:54.020442 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:54.020319 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:12:54.020442 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:54.020384 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls podName:10afb4aa-18a9-4ef8-ab02-295de0442376 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:55.02036821 +0000 UTC m=+257.779229102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls") pod "cluster-samples-operator-6748489656-hf9bw" (UID: "10afb4aa-18a9-4ef8-ab02-295de0442376") : secret "samples-operator-tls" not found Jul 28 17:12:54.434049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.433506 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" event={"ID":"358e9a0a-b26b-4b6a-b53c-2d83fea72f48","Type":"ContainerStarted","Data":"394eec65642f0fb44ab5ab6b4de2919f58570a1fa3e95199d35fd69eebade3ce"} Jul 28 17:12:54.434049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.433577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" event={"ID":"358e9a0a-b26b-4b6a-b53c-2d83fea72f48","Type":"ContainerStarted","Data":"4b65cc6f0684594846319da270ee765ec5f79098d6e9efc474b52dca07cb8dcb"} Jul 28 17:12:54.435941 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.435864 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" event={"ID":"1d65527e-a4a8-465f-81c0-2bd8b441d91b","Type":"ContainerStarted","Data":"aca24e1b1093c2394a9b20258d78b85c98115efa7b782a8121ae3d4356c4ef64"} Jul 28 17:12:54.437798 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.437771 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" event={"ID":"46326b21-01e9-4e63-85c8-516ea6d5d862","Type":"ContainerStarted","Data":"be12ca8e8037efddcf3330961d1c27bb20b91c6a5c5b6daaa68ac38401b76b50"} Jul 28 17:12:54.439820 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.439787 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" event={"ID":"b1525dd4-496e-4b33-964e-db81ff558ed9","Type":"ContainerStarted","Data":"6ab73362f56e88bce0709eb2101bce1dcf9d6573837daa4f7a86b039cf5c8e8a"} Jul 28 17:12:54.441381 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.441335 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-z6qvn" event={"ID":"a5530c61-043a-404e-afad-43651d677628","Type":"ContainerStarted","Data":"078b50b90f56fa5cbccfcede693bce0c85cd8a9f00f67f4a355f15e1f46de8b8"} Jul 28 17:12:54.453768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.452909 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-q247t" podStartSLOduration=1.4528919949999999 podStartE2EDuration="1.452891995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:54.452073482 +0000 UTC m=+257.210934398" watchObservedRunningTime="2026-07-28 17:12:54.452891995 +0000 UTC m=+257.211752909" Jul 28 17:12:54.834783 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.834741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:54.834985 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:54.834900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:54.835066 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:54.835036 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:54.835602 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:54.835101 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls podName:cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:56.835082598 +0000 UTC m=+259.593943513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-thnp6" (UID: "cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:54.835602 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:54.835505 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:12:54.835602 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:54.835555 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert podName:3b79145a-1a31-4970-82af-1212de987dc9 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:56.835537981 +0000 UTC m=+259.594398881 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9522p" (UID: "3b79145a-1a31-4970-82af-1212de987dc9") : secret "networking-console-plugin-cert" not found Jul 28 17:12:55.037654 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:55.037600 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:55.037823 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:55.037766 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:12:55.037909 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:55.037870 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls podName:10afb4aa-18a9-4ef8-ab02-295de0442376 nodeName:}" failed. No retries permitted until 2026-07-28 17:12:57.037816475 +0000 UTC m=+259.796677372 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls") pod "cluster-samples-operator-6748489656-hf9bw" (UID: "10afb4aa-18a9-4ef8-ab02-295de0442376") : secret "samples-operator-tls" not found Jul 28 17:12:55.445709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:55.445589 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" event={"ID":"1d65527e-a4a8-465f-81c0-2bd8b441d91b","Type":"ContainerStarted","Data":"bea60e7b0c782c23b831b67191a649a35f43639ea28088cdf8ec97493716a7d7"} Jul 28 17:12:55.464060 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:55.464012 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-qscf7" podStartSLOduration=1.053376654 podStartE2EDuration="2.463994796s" totalImagesPullingTime="1.410618142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:55.463076063 +0000 UTC m=+258.221936977" watchObservedRunningTime="2026-07-28 17:12:55.463994796 +0000 UTC m=+258.222855687" Jul 28 17:12:56.852432 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:56.852380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:12:56.852808 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:56.852441 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:12:56.852808 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:56.852533 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:12:56.852808 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:56.852538 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:56.852808 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:56.852594 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert podName:3b79145a-1a31-4970-82af-1212de987dc9 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:00.85258058 +0000 UTC m=+263.611441473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9522p" (UID: "3b79145a-1a31-4970-82af-1212de987dc9") : secret "networking-console-plugin-cert" not found Jul 28 17:12:56.852808 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:56.852616 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls podName:cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:00.852601878 +0000 UTC m=+263.611462771 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-thnp6" (UID: "cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:12:57.053504 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.053462 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:12:57.053683 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:57.053607 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:12:57.053683 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:57.053666 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls podName:10afb4aa-18a9-4ef8-ab02-295de0442376 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:01.053652544 +0000 UTC m=+263.812513441 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls") pod "cluster-samples-operator-6748489656-hf9bw" (UID: "10afb4aa-18a9-4ef8-ab02-295de0442376") : secret "samples-operator-tls" not found Jul 28 17:12:57.453429 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.453386 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" event={"ID":"46326b21-01e9-4e63-85c8-516ea6d5d862","Type":"ContainerStarted","Data":"be249719f4fc71d70e1b8b9cbe9fe5b50b0221f0fdf7e6addcc40cf1ff8301e4"} Jul 28 17:12:57.454979 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.454952 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" event={"ID":"b1525dd4-496e-4b33-964e-db81ff558ed9","Type":"ContainerStarted","Data":"e73d5c10f3ab30b6706244523c95866bb5bba4c750736d134e898548b4151bf5"} Jul 28 17:12:57.456707 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.456684 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-z6qvn" event={"ID":"a5530c61-043a-404e-afad-43651d677628","Type":"ContainerStarted","Data":"fa2ade88079234576f147b899c09ff441310049fdf337e0526e0a95699632b67"} Jul 28 17:12:57.472194 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.472150 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" podStartSLOduration=1.385409073 podStartE2EDuration="4.472137629s" totalImagesPullingTime="3.086728556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:57.471904683 +0000 UTC m=+260.230765600" watchObservedRunningTime="2026-07-28 17:12:57.472137629 +0000 UTC m=+260.230998543" Jul 28 17:12:57.489128 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.489079 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" podStartSLOduration=1.3218241339999999 podStartE2EDuration="4.489061516s" totalImagesPullingTime="3.167237382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:57.488970588 +0000 UTC m=+260.247831502" watchObservedRunningTime="2026-07-28 17:12:57.489061516 +0000 UTC m=+260.247922432" Jul 28 17:12:57.510421 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.510364 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-z6qvn" podStartSLOduration=1.236327927 podStartE2EDuration="4.51034527s" totalImagesPullingTime="3.274017343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:12:57.509298963 +0000 UTC m=+260.268159876" watchObservedRunningTime="2026-07-28 17:12:57.51034527 +0000 UTC m=+260.269206186" Jul 28 17:12:57.943048 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.943018 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pr6k7"] Jul 28 17:12:57.945779 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.945764 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:57.951161 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.951138 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8tp7z\"" Jul 28 17:12:57.955004 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.954986 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 17:12:57.955280 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:57.955268 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 17:12:58.013814 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.013783 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pr6k7"] Jul 28 17:12:58.062357 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.062323 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnl8v\" (UniqueName: \"kubernetes.io/projected/2321871d-b9e6-48c0-be39-201edbf8906e-kube-api-access-jnl8v\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.062512 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.062363 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.062512 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.062402 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2321871d-b9e6-48c0-be39-201edbf8906e-crio-socket\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.062512 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.062458 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.062639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.062516 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2321871d-b9e6-48c0-be39-201edbf8906e-data-volume\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.163360 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.163324 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2321871d-b9e6-48c0-be39-201edbf8906e-crio-socket\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.163360 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.163358 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.163547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.163394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2321871d-b9e6-48c0-be39-201edbf8906e-data-volume\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.163547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.163444 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2321871d-b9e6-48c0-be39-201edbf8906e-crio-socket\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.163547 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.163467 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:12:58.163547 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.163519 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:12:58.663503542 +0000 UTC m=+261.422364433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : secret "insights-runtime-extractor-tls" not found Jul 28 17:12:58.163547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.163470 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnl8v\" (UniqueName: \"kubernetes.io/projected/2321871d-b9e6-48c0-be39-201edbf8906e-kube-api-access-jnl8v\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.163710 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.163558 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.163710 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.163618 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:12:58.163710 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.163646 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:12:58.663638821 +0000 UTC m=+261.422499713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : configmap "kube-rbac-proxy" not found Jul 28 17:12:58.163869 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.163847 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2321871d-b9e6-48c0-be39-201edbf8906e-data-volume\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.179428 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.179396 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnl8v\" (UniqueName: \"kubernetes.io/projected/2321871d-b9e6-48c0-be39-201edbf8906e-kube-api-access-jnl8v\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.476695 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.476661 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8"] Jul 28 17:12:58.478936 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.478895 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" Jul 28 17:12:58.482165 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.482141 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nbd6d\"" Jul 28 17:12:58.482165 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.482159 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 28 17:12:58.482326 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.482217 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 28 17:12:58.493905 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.493881 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8"] Jul 28 17:12:58.567660 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.567612 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4dbk\" (UniqueName: \"kubernetes.io/projected/1180225f-124f-4b95-877d-de5695356f61-kube-api-access-k4dbk\") pod \"migrator-6865c9967-nvvw8\" (UID: \"1180225f-124f-4b95-877d-de5695356f61\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" Jul 28 17:12:58.668764 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.668723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.668938 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.668893 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:12:58.668938 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.668908 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4dbk\" (UniqueName: \"kubernetes.io/projected/1180225f-124f-4b95-877d-de5695356f61-kube-api-access-k4dbk\") pod \"migrator-6865c9967-nvvw8\" (UID: \"1180225f-124f-4b95-877d-de5695356f61\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" Jul 28 17:12:58.668938 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.668932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:58.669089 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.668985 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:12:59.668968692 +0000 UTC m=+262.427829587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : secret "insights-runtime-extractor-tls" not found Jul 28 17:12:58.669089 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.668995 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:12:58.669089 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:58.669047 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:12:59.669035239 +0000 UTC m=+262.427896131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : configmap "kube-rbac-proxy" not found Jul 28 17:12:58.705155 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.705120 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4dbk\" (UniqueName: \"kubernetes.io/projected/1180225f-124f-4b95-877d-de5695356f61-kube-api-access-k4dbk\") pod \"migrator-6865c9967-nvvw8\" (UID: \"1180225f-124f-4b95-877d-de5695356f61\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" Jul 28 17:12:58.791407 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.791315 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" Jul 28 17:12:58.921122 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:58.921088 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8"] Jul 28 17:12:58.924205 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:12:58.924174 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1180225f_124f_4b95_877d_de5695356f61.slice/crio-10ca699d011416305b9472eafac3d70e1518369f17e3fefab504483889c9d3ef WatchSource:0}: Error finding container 10ca699d011416305b9472eafac3d70e1518369f17e3fefab504483889c9d3ef: Status 404 returned error can't find the container with id 10ca699d011416305b9472eafac3d70e1518369f17e3fefab504483889c9d3ef Jul 28 17:12:59.465014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:59.464975 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" event={"ID":"1180225f-124f-4b95-877d-de5695356f61","Type":"ContainerStarted","Data":"10ca699d011416305b9472eafac3d70e1518369f17e3fefab504483889c9d3ef"} Jul 28 17:12:59.676370 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:59.676335 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:59.676514 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:12:59.676378 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:12:59.676514 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:59.676458 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:12:59.676619 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:59.676550 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:12:59.676619 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:59.676551 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:01.676530644 +0000 UTC m=+264.435391535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : configmap "kube-rbac-proxy" not found Jul 28 17:12:59.676619 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:12:59.676596 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:01.676585665 +0000 UTC m=+264.435446564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:00.110056 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:00.110033 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vdrdq_48646f83-5284-4e24-8ea7-da85a74922aa/dns-node-resolver/0.log" Jul 28 17:13:00.469692 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:00.469654 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" event={"ID":"1180225f-124f-4b95-877d-de5695356f61","Type":"ContainerStarted","Data":"3ac46b5630c0a185b18d4d99923721ebf2037c733be1f176b82b6c9b5a69aa7e"} Jul 28 17:13:00.470113 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:00.469707 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" event={"ID":"1180225f-124f-4b95-877d-de5695356f61","Type":"ContainerStarted","Data":"85944c5bc7dcedad78b558260f495336b7cf6ed9aeaa6195015da165272d65a0"} Jul 28 17:13:00.491613 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:00.491564 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-nvvw8" podStartSLOduration=1.408593106 podStartE2EDuration="2.491549141s" totalImagesPullingTime="1.082956035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:00.489967859 +0000 UTC m=+263.248828788" watchObservedRunningTime="2026-07-28 17:13:00.491549141 +0000 UTC m=+263.250410054" Jul 28 17:13:00.886816 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:00.886779 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:13:00.887001 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:00.886848 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:13:00.887001 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:00.886945 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:13:00.887079 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:00.886994 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:13:00.887079 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:00.887010 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert podName:3b79145a-1a31-4970-82af-1212de987dc9 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:08.886995277 +0000 UTC m=+271.645856169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9522p" (UID: "3b79145a-1a31-4970-82af-1212de987dc9") : secret "networking-console-plugin-cert" not found Jul 28 17:13:00.887079 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:00.887044 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls podName:cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:08.887032532 +0000 UTC m=+271.645893424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-thnp6" (UID: "cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:13:01.088605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:01.088571 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:13:01.088772 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:01.088729 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 28 17:13:01.088814 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:01.088799 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls podName:10afb4aa-18a9-4ef8-ab02-295de0442376 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:09.08878049 +0000 UTC m=+271.847641398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls") pod "cluster-samples-operator-6748489656-hf9bw" (UID: "10afb4aa-18a9-4ef8-ab02-295de0442376") : secret "samples-operator-tls" not found Jul 28 17:13:01.110078 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:01.110050 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vbj6k_ba17a781-9e7c-4ec2-ab60-9a2d4132c464/node-ca/0.log" Jul 28 17:13:01.694480 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:01.694440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:01.694911 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:01.694490 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:01.694911 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:01.694601 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:13:01.694911 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:01.694670 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:05.694649578 +0000 UTC m=+268.453510472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : configmap "kube-rbac-proxy" not found Jul 28 17:13:01.694911 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:01.694740 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:13:01.694911 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:01.694782 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:05.694771247 +0000 UTC m=+268.453632146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:01.910216 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:01.910187 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r2rw6_c00ba8b0-d3ea-4a1c-8967-6d48517b3568/serve-healthcheck-canary/0.log" Jul 28 17:13:05.730559 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:05.730522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:05.730986 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:05.730570 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:05.730986 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:05.730638 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:13:05.730986 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:05.730653 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:13:05.730986 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:05.730697 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:13.730681997 +0000 UTC m=+276.489542890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : configmap "kube-rbac-proxy" not found Jul 28 17:13:05.730986 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:05.730764 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:13.730745338 +0000 UTC m=+276.489606234 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:08.957691 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:08.957648 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:13:08.957691 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:08.957699 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:13:08.958131 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:08.957781 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 28 17:13:08.958131 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:08.957875 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls podName:cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:24.957860363 +0000 UTC m=+287.716721255 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-thnp6" (UID: "cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86") : secret "cluster-monitoring-operator-tls" not found Jul 28 17:13:08.958131 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:08.957876 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:13:08.958131 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:08.957921 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert podName:3b79145a-1a31-4970-82af-1212de987dc9 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:24.957910218 +0000 UTC m=+287.716771114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9522p" (UID: "3b79145a-1a31-4970-82af-1212de987dc9") : secret "networking-console-plugin-cert" not found Jul 28 17:13:09.159191 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:09.159148 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:13:09.161476 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:09.161447 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/10afb4aa-18a9-4ef8-ab02-295de0442376-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hf9bw\" (UID: \"10afb4aa-18a9-4ef8-ab02-295de0442376\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:13:09.257126 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:09.257031 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" Jul 28 17:13:09.382283 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:09.382253 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw"] Jul 28 17:13:09.499604 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:09.499573 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" event={"ID":"10afb4aa-18a9-4ef8-ab02-295de0442376","Type":"ContainerStarted","Data":"9a787df9ee0d53e1bdd70f7fe729e61399bd2b28a288a21e97b91ab6fbe0c4e9"} Jul 28 17:13:11.506784 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:11.506754 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" event={"ID":"10afb4aa-18a9-4ef8-ab02-295de0442376","Type":"ContainerStarted","Data":"deb172ffb2bc6bd3b7e211fbbeb0ec71b42c5283e6f9ae6298e64715c620ca69"} Jul 28 17:13:12.513435 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:12.513399 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" event={"ID":"10afb4aa-18a9-4ef8-ab02-295de0442376","Type":"ContainerStarted","Data":"1550ec9a1a1c5bdb115f9670d782d12f3e582f13d19698c735a8c9b3dd05d41a"} Jul 28 17:13:12.532052 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:12.532001 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hf9bw" podStartSLOduration=17.563251621 podStartE2EDuration="19.531985335s" totalImagesPullingTime="1.968733714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:12.5316554 +0000 UTC m=+275.290516314" watchObservedRunningTime="2026-07-28 17:13:12.531985335 +0000 UTC m=+275.290846307" Jul 28 17:13:13.801965 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:13.801925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:13.801965 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:13.801975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:13.802541 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:13.802120 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:13:13.802541 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:13.802186 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls podName:2321871d-b9e6-48c0-be39-201edbf8906e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:29.802166376 +0000 UTC m=+292.561027283 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pr6k7" (UID: "2321871d-b9e6-48c0-be39-201edbf8906e") : secret "insights-runtime-extractor-tls" not found Jul 28 17:13:13.802541 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:13.802470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2321871d-b9e6-48c0-be39-201edbf8906e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:19.208218 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:19.208175 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mjf49" podUID="36520022-e419-412e-bf8b-d24c5da77c7d" Jul 28 17:13:19.208218 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:19.208191 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-587668d4b-dp65p" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" Jul 28 17:13:19.534963 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:19.534881 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:13:19.535106 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:19.534881 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjf49" Jul 28 17:13:22.577016 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.576979 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:13:22.577016 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.577020 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:13:22.579433 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.579403 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36520022-e419-412e-bf8b-d24c5da77c7d-metrics-tls\") pod \"dns-default-mjf49\" (UID: \"36520022-e419-412e-bf8b-d24c5da77c7d\") " pod="openshift-dns/dns-default-mjf49" Jul 28 17:13:22.579539 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.579466 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"image-registry-587668d4b-dp65p\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:13:22.838716 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.838678 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-brvwl\"" Jul 28 17:13:22.839805 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.839787 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hwxn2\"" Jul 28 17:13:22.845972 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.845953 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:13:22.846080 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:22.846040 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mjf49" Jul 28 17:13:23.005815 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:23.005779 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-587668d4b-dp65p"] Jul 28 17:13:23.009733 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:23.009707 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaf01f69_9d9b_4bc5_ba76_47c2fe585884.slice/crio-0bd23227d4da581a9874584ff6080e1f9edf32be29378d425357f86a327faa2e WatchSource:0}: Error finding container 0bd23227d4da581a9874584ff6080e1f9edf32be29378d425357f86a327faa2e: Status 404 returned error can't find the container with id 0bd23227d4da581a9874584ff6080e1f9edf32be29378d425357f86a327faa2e Jul 28 17:13:23.018513 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:23.018411 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mjf49"] Jul 28 17:13:23.021272 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:23.021249 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36520022_e419_412e_bf8b_d24c5da77c7d.slice/crio-5dd302176ae90613e4fe88a6edcd43e6f5c6d1ffd692337be93c735f4fc33420 WatchSource:0}: Error finding container 5dd302176ae90613e4fe88a6edcd43e6f5c6d1ffd692337be93c735f4fc33420: Status 404 returned error can't find the container with id 5dd302176ae90613e4fe88a6edcd43e6f5c6d1ffd692337be93c735f4fc33420 Jul 28 17:13:23.546815 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:23.546779 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-587668d4b-dp65p" event={"ID":"baf01f69-9d9b-4bc5-ba76-47c2fe585884","Type":"ContainerStarted","Data":"176a0cf491e739063b9b46d1d45b4d74a8682e3c5a05ebdb786c0807a9dd1f7e"} Jul 28 17:13:23.547009 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:23.546921 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-587668d4b-dp65p" event={"ID":"baf01f69-9d9b-4bc5-ba76-47c2fe585884","Type":"ContainerStarted","Data":"0bd23227d4da581a9874584ff6080e1f9edf32be29378d425357f86a327faa2e"} Jul 28 17:13:23.547009 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:23.546957 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:13:23.547969 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:23.547945 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjf49" event={"ID":"36520022-e419-412e-bf8b-d24c5da77c7d","Type":"ContainerStarted","Data":"5dd302176ae90613e4fe88a6edcd43e6f5c6d1ffd692337be93c735f4fc33420"} Jul 28 17:13:23.570445 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:23.570398 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-587668d4b-dp65p" podStartSLOduration=284.57038346 podStartE2EDuration="4m44.57038346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:23.569614425 +0000 UTC m=+286.328475340" watchObservedRunningTime="2026-07-28 17:13:23.57038346 +0000 UTC m=+286.329244374" Jul 28 17:13:24.998894 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:24.998857 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:13:24.999393 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:24.998900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:13:25.001635 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:25.001606 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3b79145a-1a31-4970-82af-1212de987dc9-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9522p\" (UID: \"3b79145a-1a31-4970-82af-1212de987dc9\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:13:25.002150 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:25.002115 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-thnp6\" (UID: \"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:13:25.254928 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:25.254822 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" Jul 28 17:13:25.271812 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:25.271782 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" Jul 28 17:13:25.605674 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:25.605647 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9522p"] Jul 28 17:13:25.610205 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:25.610177 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b79145a_1a31_4970_82af_1212de987dc9.slice/crio-b3c814a8b24f732ab752df004808504d428a60bb4089afcf1ea10890c3fe6190 WatchSource:0}: Error finding container b3c814a8b24f732ab752df004808504d428a60bb4089afcf1ea10890c3fe6190: Status 404 returned error can't find the container with id b3c814a8b24f732ab752df004808504d428a60bb4089afcf1ea10890c3fe6190 Jul 28 17:13:25.621322 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:25.621251 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6"] Jul 28 17:13:25.624229 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:25.624201 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf74e9e9_bb2e_4aa6_bdad_bd1525a43b86.slice/crio-60c39d826fc7f5090efdc5e26da6077100af681752ef860c52d264a5625e7af7 WatchSource:0}: Error finding container 60c39d826fc7f5090efdc5e26da6077100af681752ef860c52d264a5625e7af7: Status 404 returned error can't find the container with id 60c39d826fc7f5090efdc5e26da6077100af681752ef860c52d264a5625e7af7 Jul 28 17:13:26.561189 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:26.561156 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" event={"ID":"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86","Type":"ContainerStarted","Data":"60c39d826fc7f5090efdc5e26da6077100af681752ef860c52d264a5625e7af7"} Jul 28 17:13:26.563179 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:26.563154 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjf49" event={"ID":"36520022-e419-412e-bf8b-d24c5da77c7d","Type":"ContainerStarted","Data":"a63ddcd988e766f740ca0432f11749df5e7af1934bbaf63f2e3b5ace8c6026a4"} Jul 28 17:13:26.563314 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:26.563202 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mjf49" event={"ID":"36520022-e419-412e-bf8b-d24c5da77c7d","Type":"ContainerStarted","Data":"7bf9212598c3b61c4e9005cd4445c978099e725ff79415a683bf81761699e480"} Jul 28 17:13:26.563314 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:26.563263 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mjf49" Jul 28 17:13:26.564518 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:26.564495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" event={"ID":"3b79145a-1a31-4970-82af-1212de987dc9","Type":"ContainerStarted","Data":"b3c814a8b24f732ab752df004808504d428a60bb4089afcf1ea10890c3fe6190"} Jul 28 17:13:26.582598 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:26.582551 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mjf49" podStartSLOduration=252.084991689 podStartE2EDuration="4m14.582537431s" totalImagesPullingTime="2.497545742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:26.581678303 +0000 UTC m=+289.340539221" watchObservedRunningTime="2026-07-28 17:13:26.582537431 +0000 UTC m=+289.341398345" Jul 28 17:13:27.569698 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:27.569632 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" event={"ID":"3b79145a-1a31-4970-82af-1212de987dc9","Type":"ContainerStarted","Data":"b473a9610f2e61a2a94a2c5a00e11421a33b3ee39092dfb841e06438169d23f1"} Jul 28 17:13:27.571808 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:27.571783 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" event={"ID":"cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86","Type":"ContainerStarted","Data":"9584dfeac2aab88b49dd1bb8cad6e5c40c774a13c16f52e8cb0c687bcf31790b"} Jul 28 17:13:27.588892 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:27.588842 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9522p" podStartSLOduration=32.704782192 podStartE2EDuration="34.588811715s" totalImagesPullingTime="1.884029523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:27.587549669 +0000 UTC m=+290.346410596" watchObservedRunningTime="2026-07-28 17:13:27.588811715 +0000 UTC m=+290.347672630" Jul 28 17:13:27.608937 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:27.608818 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-thnp6" podStartSLOduration=32.739010099 podStartE2EDuration="34.608796188s" totalImagesPullingTime="1.869786089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:27.608667768 +0000 UTC m=+290.367528683" watchObservedRunningTime="2026-07-28 17:13:27.608796188 +0000 UTC m=+290.367657105" Jul 28 17:13:28.042513 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.042474 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw"] Jul 28 17:13:28.047153 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.047128 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:28.050403 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.050381 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-m6gjq\"" Jul 28 17:13:28.050521 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.050382 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 17:13:28.054282 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.054262 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw"] Jul 28 17:13:28.227559 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.227528 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c435ba02-2d0b-4e5b-9d28-91ebae5043b3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-n4qrw\" (UID: \"c435ba02-2d0b-4e5b-9d28-91ebae5043b3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:28.328319 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.328222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c435ba02-2d0b-4e5b-9d28-91ebae5043b3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-n4qrw\" (UID: \"c435ba02-2d0b-4e5b-9d28-91ebae5043b3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:28.328468 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:28.328355 2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 28 17:13:28.328468 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:28.328423 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c435ba02-2d0b-4e5b-9d28-91ebae5043b3-tls-certificates podName:c435ba02-2d0b-4e5b-9d28-91ebae5043b3 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:28.828405698 +0000 UTC m=+291.587266591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/c435ba02-2d0b-4e5b-9d28-91ebae5043b3-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" (UID: "c435ba02-2d0b-4e5b-9d28-91ebae5043b3") : secret "prometheus-operator-admission-webhook-tls" not found Jul 28 17:13:28.832243 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.832202 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c435ba02-2d0b-4e5b-9d28-91ebae5043b3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-n4qrw\" (UID: \"c435ba02-2d0b-4e5b-9d28-91ebae5043b3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:28.834663 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.834639 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c435ba02-2d0b-4e5b-9d28-91ebae5043b3-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-n4qrw\" (UID: \"c435ba02-2d0b-4e5b-9d28-91ebae5043b3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:28.956814 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:28.956758 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:29.084401 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:29.084367 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw"] Jul 28 17:13:29.088153 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:29.088127 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc435ba02_2d0b_4e5b_9d28_91ebae5043b3.slice/crio-0687a776c3168d367b03443999457a5a0f6e4842930a3ce415ab464d9a47df31 WatchSource:0}: Error finding container 0687a776c3168d367b03443999457a5a0f6e4842930a3ce415ab464d9a47df31: Status 404 returned error can't find the container with id 0687a776c3168d367b03443999457a5a0f6e4842930a3ce415ab464d9a47df31 Jul 28 17:13:29.580223 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:29.580135 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" event={"ID":"c435ba02-2d0b-4e5b-9d28-91ebae5043b3","Type":"ContainerStarted","Data":"0687a776c3168d367b03443999457a5a0f6e4842930a3ce415ab464d9a47df31"} Jul 28 17:13:29.840983 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:29.840944 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:29.843228 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:29.843200 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2321871d-b9e6-48c0-be39-201edbf8906e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pr6k7\" (UID: \"2321871d-b9e6-48c0-be39-201edbf8906e\") " pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:30.054892 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:30.054859 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pr6k7" Jul 28 17:13:30.219625 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:30.219592 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pr6k7"] Jul 28 17:13:30.498394 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:30.498322 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2321871d_b9e6_48c0_be39_201edbf8906e.slice/crio-bae80221bf8873fdea8b7599ba94430b4fd995f15a6c3eed8abf64750876211d WatchSource:0}: Error finding container bae80221bf8873fdea8b7599ba94430b4fd995f15a6c3eed8abf64750876211d: Status 404 returned error can't find the container with id bae80221bf8873fdea8b7599ba94430b4fd995f15a6c3eed8abf64750876211d Jul 28 17:13:30.584494 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:30.584455 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pr6k7" event={"ID":"2321871d-b9e6-48c0-be39-201edbf8906e","Type":"ContainerStarted","Data":"bae80221bf8873fdea8b7599ba94430b4fd995f15a6c3eed8abf64750876211d"} Jul 28 17:13:31.589549 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:31.589511 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" event={"ID":"c435ba02-2d0b-4e5b-9d28-91ebae5043b3","Type":"ContainerStarted","Data":"b210adbacf0ce510fc2a5c7d0d01c9443a1f9c2f59fd25ad5a2b30120a681b80"} Jul 28 17:13:31.590040 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:31.589676 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:31.595163 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:31.595137 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" Jul 28 17:13:31.607478 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:31.607417 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-n4qrw" podStartSLOduration=1.8454041079999999 podStartE2EDuration="3.607398631s" totalImagesPullingTime="1.761994523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:31.60652034 +0000 UTC m=+294.365381255" watchObservedRunningTime="2026-07-28 17:13:31.607398631 +0000 UTC m=+294.366259546" Jul 28 17:13:32.105639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.105604 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-pkgcg"] Jul 28 17:13:32.119521 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.119495 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-pkgcg"] Jul 28 17:13:32.119656 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.119590 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.124045 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.124015 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 28 17:13:32.124184 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.124020 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 28 17:13:32.124184 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.124020 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 17:13:32.124184 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.124077 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vxfgg\"" Jul 28 17:13:32.162880 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.162846 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.163026 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.162895 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5003b68a-bbf3-4382-b934-7c5371c76bcc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.163026 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.162924 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs8wr\" (UniqueName: \"kubernetes.io/projected/5003b68a-bbf3-4382-b934-7c5371c76bcc-kube-api-access-gs8wr\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.163026 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.162955 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.263937 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.263903 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.264143 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.263966 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5003b68a-bbf3-4382-b934-7c5371c76bcc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.264143 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.264018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs8wr\" (UniqueName: \"kubernetes.io/projected/5003b68a-bbf3-4382-b934-7c5371c76bcc-kube-api-access-gs8wr\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.264143 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.264068 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.264298 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:32.264187 2570 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 28 17:13:32.264298 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:32.264246 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-tls podName:5003b68a-bbf3-4382-b934-7c5371c76bcc nodeName:}" failed. No retries permitted until 2026-07-28 17:13:32.764227338 +0000 UTC m=+295.523088230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-tls") pod "prometheus-operator-84d69f794d-pkgcg" (UID: "5003b68a-bbf3-4382-b934-7c5371c76bcc") : secret "prometheus-operator-tls" not found Jul 28 17:13:32.264737 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.264718 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5003b68a-bbf3-4382-b934-7c5371c76bcc-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.266945 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.266918 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.273911 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.273885 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs8wr\" (UniqueName: \"kubernetes.io/projected/5003b68a-bbf3-4382-b934-7c5371c76bcc-kube-api-access-gs8wr\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.719395 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:32.719361 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-plwl2" podUID="b00417f8-bed7-4fcb-b3ec-f912fc8dfb91" Jul 28 17:13:32.768643 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.768610 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:32.771693 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:32.771669 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/5003b68a-bbf3-4382-b934-7c5371c76bcc-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-pkgcg\" (UID: \"5003b68a-bbf3-4382-b934-7c5371c76bcc\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:33.029035 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:33.028951 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" Jul 28 17:13:33.163505 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:33.163472 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-pkgcg"] Jul 28 17:13:35.359661 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:35.359623 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5003b68a_bbf3_4382_b934_7c5371c76bcc.slice/crio-aed0768112778df3532307153745684fc9aa0334b116803919cbc0be127109c6 WatchSource:0}: Error finding container aed0768112778df3532307153745684fc9aa0334b116803919cbc0be127109c6: Status 404 returned error can't find the container with id aed0768112778df3532307153745684fc9aa0334b116803919cbc0be127109c6 Jul 28 17:13:35.605278 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:35.605227 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" event={"ID":"5003b68a-bbf3-4382-b934-7c5371c76bcc","Type":"ContainerStarted","Data":"aed0768112778df3532307153745684fc9aa0334b116803919cbc0be127109c6"} Jul 28 17:13:36.574757 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:36.574723 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mjf49" Jul 28 17:13:36.613199 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:36.613126 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pr6k7" event={"ID":"2321871d-b9e6-48c0-be39-201edbf8906e","Type":"ContainerStarted","Data":"dc9084de03246584caf8c8cded0e9fd445021ffd76c5ccdbec6362bc186c9de2"} Jul 28 17:13:37.619489 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:37.619446 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" event={"ID":"5003b68a-bbf3-4382-b934-7c5371c76bcc","Type":"ContainerStarted","Data":"06bfd4281404ac15e2428de571ae2019832ffb8c0b3236d5d1b96c2cac04a3ce"} Jul 28 17:13:37.620107 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:37.620078 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" event={"ID":"5003b68a-bbf3-4382-b934-7c5371c76bcc","Type":"ContainerStarted","Data":"60217ba7b692c15a4f8c7f198fb97ccb6f8fe32da278bed94fe43eab56f0769e"} Jul 28 17:13:37.642349 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:37.642162 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-pkgcg" podStartSLOduration=4.054455568 podStartE2EDuration="5.642145798s" totalImagesPullingTime="1.58769023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:37.641195338 +0000 UTC m=+300.400056255" watchObservedRunningTime="2026-07-28 17:13:37.642145798 +0000 UTC m=+300.401006758" Jul 28 17:13:39.249237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.249212 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 17:13:39.337113 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.337093 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:13:39.475514 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.475456 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5mdxh"] Jul 28 17:13:39.478731 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.478714 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.481450 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.481425 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 17:13:39.482012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.481992 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 17:13:39.482124 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.482027 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 17:13:39.482241 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.482225 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m2slg\"" Jul 28 17:13:39.525946 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.525865 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-wtmp\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.525946 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.525931 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-root\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.526116 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.525952 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b4c68a3-7f4e-4048-a86a-e6163167754e-metrics-client-ca\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.526116 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.525970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-sys\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.526116 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.525984 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-textfile\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.526116 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.526000 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.526116 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.526052 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.526116 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.526086 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhg5h\" (UniqueName: \"kubernetes.io/projected/0b4c68a3-7f4e-4048-a86a-e6163167754e-kube-api-access-rhg5h\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.526301 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.526135 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-tls\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.626755 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626724 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-root\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.626755 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626764 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b4c68a3-7f4e-4048-a86a-e6163167754e-metrics-client-ca\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626791 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-sys\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626792 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-root\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626810 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-textfile\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626868 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626880 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-sys\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626906 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626933 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhg5h\" (UniqueName: \"kubernetes.io/projected/0b4c68a3-7f4e-4048-a86a-e6163167754e-kube-api-access-rhg5h\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.626967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-tls\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.627014 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-wtmp\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627410 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.627163 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-wtmp\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627410 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.627171 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-textfile\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627410 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:39.627210 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 17:13:39.627410 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:39.627282 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-tls podName:0b4c68a3-7f4e-4048-a86a-e6163167754e nodeName:}" failed. No retries permitted until 2026-07-28 17:13:40.127261618 +0000 UTC m=+302.886122526 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-tls") pod "node-exporter-5mdxh" (UID: "0b4c68a3-7f4e-4048-a86a-e6163167754e") : secret "node-exporter-tls" not found Jul 28 17:13:39.627576 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.627516 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b4c68a3-7f4e-4048-a86a-e6163167754e-metrics-client-ca\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.627576 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.627516 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-accelerators-collector-config\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.628879 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.628854 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pr6k7" event={"ID":"2321871d-b9e6-48c0-be39-201edbf8906e","Type":"ContainerStarted","Data":"d704b6c0c0ffa2b5f2562e9382d8beec44facae0b29244a91009788b523c6867"} Jul 28 17:13:39.629534 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.629516 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:39.636001 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:39.635980 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhg5h\" (UniqueName: \"kubernetes.io/projected/0b4c68a3-7f4e-4048-a86a-e6163167754e-kube-api-access-rhg5h\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:40.131067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.131030 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-tls\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:40.133634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.133593 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0b4c68a3-7f4e-4048-a86a-e6163167754e-node-exporter-tls\") pod \"node-exporter-5mdxh\" (UID: \"0b4c68a3-7f4e-4048-a86a-e6163167754e\") " pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:40.387596 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.387500 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5mdxh" Jul 28 17:13:40.400043 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:40.400009 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b4c68a3_7f4e_4048_a86a_e6163167754e.slice/crio-d3014c2f4f647dca898b54cc05e615641056aebac4a7ca929ab683235014196c WatchSource:0}: Error finding container d3014c2f4f647dca898b54cc05e615641056aebac4a7ca929ab683235014196c: Status 404 returned error can't find the container with id d3014c2f4f647dca898b54cc05e615641056aebac4a7ca929ab683235014196c Jul 28 17:13:40.628446 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.628405 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:13:40.634457 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.634434 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.637387 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.637364 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 17:13:40.637387 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.637379 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 17:13:40.637729 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.637673 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mdxh" event={"ID":"0b4c68a3-7f4e-4048-a86a-e6163167754e","Type":"ContainerStarted","Data":"d3014c2f4f647dca898b54cc05e615641056aebac4a7ca929ab683235014196c"} Jul 28 17:13:40.638855 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.638813 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 17:13:40.638955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.638816 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 17:13:40.638955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.638912 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 17:13:40.639167 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.639140 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 17:13:40.639283 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.639270 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 17:13:40.639352 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.639315 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pgprt\"" Jul 28 17:13:40.639418 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.639401 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 17:13:40.639658 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.639560 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 17:13:40.648420 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.648397 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:13:40.736027 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.735987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-web-config\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736075 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736144 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736175 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736372 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736230 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736372 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736298 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736372 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736328 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736372 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736346 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736372 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736364 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736606 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736390 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt8x8\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-kube-api-access-qt8x8\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736606 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736519 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-out\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736606 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736559 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.736606 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.736595 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.837488 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.837673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837498 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.837673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837530 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt8x8\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-kube-api-access-qt8x8\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.837673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-out\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.837673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.837673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837629 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.837673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837669 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-web-config\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.838015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837696 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.838015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837720 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.838015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.838015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837792 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.838015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.838015 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.837900 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.839393 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.838630 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.839393 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:13:40.839141 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle podName:b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2 nodeName:}" failed. No retries permitted until 2026-07-28 17:13:41.339117664 +0000 UTC m=+304.097978571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2") : configmap references non-existent config key: ca-bundle.crt Jul 28 17:13:40.841444 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.841151 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.843536 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.842577 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-out\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.843536 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.843014 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.843536 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.843283 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.843536 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.843499 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-web-config\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.845089 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.844270 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.845089 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.844742 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.845089 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.845052 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.845309 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.845287 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-volume\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.845373 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.845350 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:40.851102 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:40.851079 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt8x8\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-kube-api-access-qt8x8\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:41.343394 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:41.343354 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:41.344291 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:41.344267 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:41.550914 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:41.550878 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:13:41.618781 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:41.618741 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-587668d4b-dp65p"] Jul 28 17:13:41.624529 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:41.624468 2570 patch_prober.go:28] interesting pod/image-registry-587668d4b-dp65p container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 28 17:13:41.624678 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:41.624548 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-587668d4b-dp65p" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:13:42.543088 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.543058 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-97b7b6f55-7287d"] Jul 28 17:13:42.564424 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.564394 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-97b7b6f55-7287d"] Jul 28 17:13:42.564904 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.564551 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.567855 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.567796 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 17:13:42.568215 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.567951 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 17:13:42.568215 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.568068 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 17:13:42.568340 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.568313 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 17:13:42.568683 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.568565 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dvhp82kn4iqb9\"" Jul 28 17:13:42.568683 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.568590 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-t57l6\"" Jul 28 17:13:42.568683 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.568657 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 17:13:42.656337 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656297 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-tls\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.656510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656338 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-metrics-client-ca\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.656510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656384 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-grpc-tls\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.656510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.656510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.656510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbh5g\" (UniqueName: \"kubernetes.io/projected/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-kube-api-access-cbh5g\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.656792 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.656792 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.656624 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758198 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758154 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758394 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-tls\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758394 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758240 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-metrics-client-ca\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758394 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758266 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-grpc-tls\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758394 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758346 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758638 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758400 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758638 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758431 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbh5g\" (UniqueName: \"kubernetes.io/projected/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-kube-api-access-cbh5g\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.758638 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.758480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.759177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.759147 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-metrics-client-ca\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.761233 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.761211 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.761450 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.761426 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.761510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.761490 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.762023 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.761998 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-grpc-tls\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.762129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.762074 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-tls\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.762882 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.762864 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.768646 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.768622 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbh5g\" (UniqueName: \"kubernetes.io/projected/00e1ef9a-175d-40d0-9cf0-4a2661c4c606-kube-api-access-cbh5g\") pod \"thanos-querier-97b7b6f55-7287d\" (UID: \"00e1ef9a-175d-40d0-9cf0-4a2661c4c606\") " pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:42.876799 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:42.876758 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:43.194859 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.194716 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-97b7b6f55-7287d"] Jul 28 17:13:43.220135 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.220102 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:13:43.225702 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:43.225674 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00e1ef9a_175d_40d0_9cf0_4a2661c4c606.slice/crio-e6a864d355648438f9a57a507793fc86a236d6f7107ae9d0c73de3fbf40682aa WatchSource:0}: Error finding container e6a864d355648438f9a57a507793fc86a236d6f7107ae9d0c73de3fbf40682aa: Status 404 returned error can't find the container with id e6a864d355648438f9a57a507793fc86a236d6f7107ae9d0c73de3fbf40682aa Jul 28 17:13:43.240589 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:43.240561 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f9bdb2_5a96_41e3_bc9a_cea72e6e1bf2.slice/crio-f51e027841c3b3f73d3791e0d22b9e7da1656f0141f0c0098900d083333b2be0 WatchSource:0}: Error finding container f51e027841c3b3f73d3791e0d22b9e7da1656f0141f0c0098900d083333b2be0: Status 404 returned error can't find the container with id f51e027841c3b3f73d3791e0d22b9e7da1656f0141f0c0098900d083333b2be0 Jul 28 17:13:43.652044 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.651998 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pr6k7" event={"ID":"2321871d-b9e6-48c0-be39-201edbf8906e","Type":"ContainerStarted","Data":"93dc5dc10129085b5d2e01225980d8f6bc79f9d07c3ef45b51e1a8f19bb90802"} Jul 28 17:13:43.653158 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.653131 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerStarted","Data":"f51e027841c3b3f73d3791e0d22b9e7da1656f0141f0c0098900d083333b2be0"} Jul 28 17:13:43.654126 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.654104 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" event={"ID":"00e1ef9a-175d-40d0-9cf0-4a2661c4c606","Type":"ContainerStarted","Data":"e6a864d355648438f9a57a507793fc86a236d6f7107ae9d0c73de3fbf40682aa"} Jul 28 17:13:43.655531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.655508 2570 generic.go:356] "Generic (PLEG): container finished" podID="0b4c68a3-7f4e-4048-a86a-e6163167754e" containerID="8a3ef17756ef4052bcb874131f174d0ae6d1b83e0ad73e2a02e67f54c9bec058" exitCode=0 Jul 28 17:13:43.655612 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.655569 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mdxh" event={"ID":"0b4c68a3-7f4e-4048-a86a-e6163167754e","Type":"ContainerDied","Data":"8a3ef17756ef4052bcb874131f174d0ae6d1b83e0ad73e2a02e67f54c9bec058"} Jul 28 17:13:43.672026 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.671984 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pr6k7" podStartSLOduration=34.191239783 podStartE2EDuration="46.6719674s" totalImagesPullingTime="12.480727617s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:12:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:43.670779709 +0000 UTC m=+306.429640623" watchObservedRunningTime="2026-07-28 17:13:43.6719674 +0000 UTC m=+306.430828353" Jul 28 17:13:43.893007 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.892970 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7545c9ffff-7km2p"] Jul 28 17:13:43.896435 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.896412 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:43.899242 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.899217 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 17:13:43.899368 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.899265 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 17:13:43.899368 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.899268 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 17:13:43.899758 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.899738 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-llds6\"" Jul 28 17:13:43.899886 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.899773 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6o6d9onba7ba1\"" Jul 28 17:13:43.900051 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.900035 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 17:13:43.905507 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.905473 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7545c9ffff-7km2p"] Jul 28 17:13:43.974754 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.974717 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-secret-metrics-server-client-certs\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:43.974959 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.974763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czrwl\" (UniqueName: \"kubernetes.io/projected/639095c0-04a3-4121-99a2-3acf77a4c993-kube-api-access-czrwl\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:43.974959 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.974808 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/639095c0-04a3-4121-99a2-3acf77a4c993-audit-log\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:43.974959 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.974859 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-client-ca-bundle\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:43.975132 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.974942 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/639095c0-04a3-4121-99a2-3acf77a4c993-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:43.975132 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.974999 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-secret-metrics-server-tls\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:43.975132 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:43.975032 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/639095c0-04a3-4121-99a2-3acf77a4c993-metrics-server-audit-profiles\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.075642 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.075599 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-secret-metrics-server-client-certs\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.075841 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.075653 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czrwl\" (UniqueName: \"kubernetes.io/projected/639095c0-04a3-4121-99a2-3acf77a4c993-kube-api-access-czrwl\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.075841 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.075714 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/639095c0-04a3-4121-99a2-3acf77a4c993-audit-log\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.075841 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.075753 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-client-ca-bundle\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.075841 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.075792 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/639095c0-04a3-4121-99a2-3acf77a4c993-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.075841 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.075812 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-secret-metrics-server-tls\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.076102 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.075848 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/639095c0-04a3-4121-99a2-3acf77a4c993-metrics-server-audit-profiles\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.076280 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.076246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/639095c0-04a3-4121-99a2-3acf77a4c993-audit-log\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.076737 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.076691 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/639095c0-04a3-4121-99a2-3acf77a4c993-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.077105 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.077079 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/639095c0-04a3-4121-99a2-3acf77a4c993-metrics-server-audit-profiles\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.079230 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.079177 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-client-ca-bundle\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.079857 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.079492 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-secret-metrics-server-client-certs\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.079857 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.079799 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/639095c0-04a3-4121-99a2-3acf77a4c993-secret-metrics-server-tls\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.084445 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.084425 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czrwl\" (UniqueName: \"kubernetes.io/projected/639095c0-04a3-4121-99a2-3acf77a4c993-kube-api-access-czrwl\") pod \"metrics-server-7545c9ffff-7km2p\" (UID: \"639095c0-04a3-4121-99a2-3acf77a4c993\") " pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.208797 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.208586 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:13:44.222939 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.222851 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv"] Jul 28 17:13:44.226418 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.226392 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.229507 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.229486 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 28 17:13:44.229638 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.229556 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8wfgl\"" Jul 28 17:13:44.235488 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.235422 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv"] Jul 28 17:13:44.278263 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.278170 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/471dd41e-aae6-424e-9c09-7f24e42f012c-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-k9qnv\" (UID: \"471dd41e-aae6-424e-9c09-7f24e42f012c\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.278263 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.278229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr78m\" (UniqueName: \"kubernetes.io/projected/471dd41e-aae6-424e-9c09-7f24e42f012c-kube-api-access-dr78m\") pod \"monitoring-plugin-68b7fc9574-k9qnv\" (UID: \"471dd41e-aae6-424e-9c09-7f24e42f012c\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.378800 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.378760 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/471dd41e-aae6-424e-9c09-7f24e42f012c-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-k9qnv\" (UID: \"471dd41e-aae6-424e-9c09-7f24e42f012c\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.378967 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.378837 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr78m\" (UniqueName: \"kubernetes.io/projected/471dd41e-aae6-424e-9c09-7f24e42f012c-kube-api-access-dr78m\") pod \"monitoring-plugin-68b7fc9574-k9qnv\" (UID: \"471dd41e-aae6-424e-9c09-7f24e42f012c\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.381588 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.381550 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/471dd41e-aae6-424e-9c09-7f24e42f012c-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-k9qnv\" (UID: \"471dd41e-aae6-424e-9c09-7f24e42f012c\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.388291 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.388263 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr78m\" (UniqueName: \"kubernetes.io/projected/471dd41e-aae6-424e-9c09-7f24e42f012c-kube-api-access-dr78m\") pod \"monitoring-plugin-68b7fc9574-k9qnv\" (UID: \"471dd41e-aae6-424e-9c09-7f24e42f012c\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.526094 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.526059 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7545c9ffff-7km2p"] Jul 28 17:13:44.530542 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:13:44.530500 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639095c0_04a3_4121_99a2_3acf77a4c993.slice/crio-6afa15a909998f83e5a6e5400de467d5d7d1873e357523a7765f0de277446dcf WatchSource:0}: Error finding container 6afa15a909998f83e5a6e5400de467d5d7d1873e357523a7765f0de277446dcf: Status 404 returned error can't find the container with id 6afa15a909998f83e5a6e5400de467d5d7d1873e357523a7765f0de277446dcf Jul 28 17:13:44.546338 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.546313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:44.665510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.665393 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mdxh" event={"ID":"0b4c68a3-7f4e-4048-a86a-e6163167754e","Type":"ContainerStarted","Data":"969b835b608fb1332759eec42737d5123940b1cdaab136a002bc71e7f15d274a"} Jul 28 17:13:44.665510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.665459 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5mdxh" event={"ID":"0b4c68a3-7f4e-4048-a86a-e6163167754e","Type":"ContainerStarted","Data":"890755162f6d4e84f822d4a0a174ace08a546ba0972ec77e295af223af0a8ffa"} Jul 28 17:13:44.668266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.668091 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" event={"ID":"639095c0-04a3-4121-99a2-3acf77a4c993","Type":"ContainerStarted","Data":"6afa15a909998f83e5a6e5400de467d5d7d1873e357523a7765f0de277446dcf"} Jul 28 17:13:44.670426 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.670274 2570 generic.go:356] "Generic (PLEG): container finished" podID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerID="bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db" exitCode=0 Jul 28 17:13:44.670426 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.670439 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db"} Jul 28 17:13:44.687679 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.687615 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5mdxh" podStartSLOduration=4.695693022 podStartE2EDuration="5.687591258s" totalImagesPullingTime="914.896583ms" totalInitContainerRuntime="77.001653ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:44.685269909 +0000 UTC m=+307.444130823" watchObservedRunningTime="2026-07-28 17:13:44.687591258 +0000 UTC m=+307.446452172" Jul 28 17:13:44.726405 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:44.724601 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv"] Jul 28 17:13:45.686661 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:45.686410 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" event={"ID":"471dd41e-aae6-424e-9c09-7f24e42f012c","Type":"ContainerStarted","Data":"e91732c9602321252296d1eb092654b916c7937ff690f627022902d506084eea"} Jul 28 17:13:45.692219 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:45.691633 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" event={"ID":"00e1ef9a-175d-40d0-9cf0-4a2661c4c606","Type":"ContainerStarted","Data":"1d089828bb08817756cd194dc0046f766dee2f1ff6805954fa34cb6059afa06d"} Jul 28 17:13:45.692219 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:45.691707 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" event={"ID":"00e1ef9a-175d-40d0-9cf0-4a2661c4c606","Type":"ContainerStarted","Data":"679f17db847ac63a564f754ef31adeca393befd85c9da663c7452dfe55279d2f"} Jul 28 17:13:45.692219 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:45.691751 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" event={"ID":"00e1ef9a-175d-40d0-9cf0-4a2661c4c606","Type":"ContainerStarted","Data":"26ea6f0751cdf81f745bd9c903f0091865c01b03c6577c92c6553b4b357cbfc6"} Jul 28 17:13:45.719891 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:45.719290 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:13:47.703023 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.702993 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" event={"ID":"639095c0-04a3-4121-99a2-3acf77a4c993","Type":"ContainerStarted","Data":"b4e5a32ba9ab4f4d2130209e98a1dab1d3f7a881b4c171b94ac453398da77af6"} Jul 28 17:13:47.705882 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.705812 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" event={"ID":"471dd41e-aae6-424e-9c09-7f24e42f012c","Type":"ContainerStarted","Data":"9adaa80fd86089c2f46c916aeddcf996aca7940116bfaf066be3c3c29e9957c4"} Jul 28 17:13:47.706234 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.706218 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:47.710645 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.710623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerStarted","Data":"dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e"} Jul 28 17:13:47.712292 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.712269 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" Jul 28 17:13:47.714172 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.714136 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" event={"ID":"00e1ef9a-175d-40d0-9cf0-4a2661c4c606","Type":"ContainerStarted","Data":"e3c4aead7ef49df0176a940b67991f5b970a5770c840c1c63ca4251e96396c34"} Jul 28 17:13:47.728284 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.728233 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" podStartSLOduration=1.7192141589999999 podStartE2EDuration="4.728211915s" totalImagesPullingTime="3.008997756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:47.724857171 +0000 UTC m=+310.483718078" watchObservedRunningTime="2026-07-28 17:13:47.728211915 +0000 UTC m=+310.487072830" Jul 28 17:13:47.755958 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:47.755785 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-k9qnv" podStartSLOduration=0.945354711 podStartE2EDuration="3.755762893s" totalImagesPullingTime="2.810408182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:47.744848431 +0000 UTC m=+310.503709337" watchObservedRunningTime="2026-07-28 17:13:47.755762893 +0000 UTC m=+310.514623809" Jul 28 17:13:48.721506 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.721468 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerStarted","Data":"f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51"} Jul 28 17:13:48.722032 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.721577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerStarted","Data":"1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0"} Jul 28 17:13:48.722032 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.721622 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerStarted","Data":"c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e"} Jul 28 17:13:48.722032 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.721669 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerStarted","Data":"ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2"} Jul 28 17:13:48.722032 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.721711 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerStarted","Data":"5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109"} Jul 28 17:13:48.724370 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.724343 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" event={"ID":"00e1ef9a-175d-40d0-9cf0-4a2661c4c606","Type":"ContainerStarted","Data":"32ffe3819b8797307d6de312843f80d929685c4860f3d012355d262d58608f98"} Jul 28 17:13:48.724508 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.724420 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" event={"ID":"00e1ef9a-175d-40d0-9cf0-4a2661c4c606","Type":"ContainerStarted","Data":"239250e311649b09b2ea6ed742f5449222b99951375bb26c64bf7e472403f036"} Jul 28 17:13:48.757436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.757382 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.683738528 podStartE2EDuration="8.757365377s" totalImagesPullingTime="4.052803648s" totalInitContainerRuntime="20.823201ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:48.754593421 +0000 UTC m=+311.513454335" watchObservedRunningTime="2026-07-28 17:13:48.757365377 +0000 UTC m=+311.516226293" Jul 28 17:13:48.781659 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:48.781591 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" podStartSLOduration=2.710626246 podStartE2EDuration="6.781569644s" totalImagesPullingTime="4.070943398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:48.779449899 +0000 UTC m=+311.538310838" watchObservedRunningTime="2026-07-28 17:13:48.781569644 +0000 UTC m=+311.540430559" Jul 28 17:13:49.727937 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:49.727904 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:50.741849 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:50.741805 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-97b7b6f55-7287d" Jul 28 17:13:51.625108 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:13:51.625080 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:14:04.208903 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:04.208874 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:14:04.209392 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:04.208923 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:14:06.639931 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:06.639858 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-587668d4b-dp65p" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" containerName="registry" containerID="cri-o://176a0cf491e739063b9b46d1d45b4d74a8682e3c5a05ebdb786c0807a9dd1f7e" gracePeriod=30 Jul 28 17:14:06.799574 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:06.799542 2570 generic.go:356] "Generic (PLEG): container finished" podID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" containerID="176a0cf491e739063b9b46d1d45b4d74a8682e3c5a05ebdb786c0807a9dd1f7e" exitCode=0 Jul 28 17:14:06.799724 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:06.799617 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-587668d4b-dp65p" event={"ID":"baf01f69-9d9b-4bc5-ba76-47c2fe585884","Type":"ContainerDied","Data":"176a0cf491e739063b9b46d1d45b4d74a8682e3c5a05ebdb786c0807a9dd1f7e"} Jul 28 17:14:07.469534 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.469504 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:14:07.488711 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.488682 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-image-registry-private-configuration\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.488876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.488721 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjjvz\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-kube-api-access-tjjvz\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.488876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.488745 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-bound-sa-token\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.488876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.488776 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-certificates\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.488876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.488801 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-installation-pull-secrets\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.488876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.488821 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.489135 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.488973 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf01f69-9d9b-4bc5-ba76-47c2fe585884-ca-trust-extracted\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.489135 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.489027 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-trusted-ca\") pod \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\" (UID: \"baf01f69-9d9b-4bc5-ba76-47c2fe585884\") " Jul 28 17:14:07.489280 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.489252 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:07.489426 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.489363 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-certificates\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.489730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.489701 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:07.491366 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.491320 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:14:07.491664 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.491637 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-kube-api-access-tjjvz" (OuterVolumeSpecName: "kube-api-access-tjjvz") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "kube-api-access-tjjvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:14:07.491664 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.491645 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:07.491843 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.491734 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:07.491843 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.491769 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:14:07.499123 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.499097 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/baf01f69-9d9b-4bc5-ba76-47c2fe585884-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "baf01f69-9d9b-4bc5-ba76-47c2fe585884" (UID: "baf01f69-9d9b-4bc5-ba76-47c2fe585884"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:14:07.590364 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.590328 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/baf01f69-9d9b-4bc5-ba76-47c2fe585884-trusted-ca\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.590547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.590371 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-image-registry-private-configuration\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.590547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.590387 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjjvz\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-kube-api-access-tjjvz\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.590547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.590400 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-bound-sa-token\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.590547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.590412 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/baf01f69-9d9b-4bc5-ba76-47c2fe585884-installation-pull-secrets\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.590547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.590426 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/baf01f69-9d9b-4bc5-ba76-47c2fe585884-registry-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.590547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.590437 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/baf01f69-9d9b-4bc5-ba76-47c2fe585884-ca-trust-extracted\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:14:07.806293 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.806259 2570 generic.go:356] "Generic (PLEG): container finished" podID="a5530c61-043a-404e-afad-43651d677628" containerID="fa2ade88079234576f147b899c09ff441310049fdf337e0526e0a95699632b67" exitCode=0 Jul 28 17:14:07.806699 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.806328 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-z6qvn" event={"ID":"a5530c61-043a-404e-afad-43651d677628","Type":"ContainerDied","Data":"fa2ade88079234576f147b899c09ff441310049fdf337e0526e0a95699632b67"} Jul 28 17:14:07.806753 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.806725 2570 scope.go:117] "RemoveContainer" containerID="fa2ade88079234576f147b899c09ff441310049fdf337e0526e0a95699632b67" Jul 28 17:14:07.807694 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.807679 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-587668d4b-dp65p" Jul 28 17:14:07.807791 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.807687 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-587668d4b-dp65p" event={"ID":"baf01f69-9d9b-4bc5-ba76-47c2fe585884","Type":"ContainerDied","Data":"0bd23227d4da581a9874584ff6080e1f9edf32be29378d425357f86a327faa2e"} Jul 28 17:14:07.807791 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.807743 2570 scope.go:117] "RemoveContainer" containerID="176a0cf491e739063b9b46d1d45b4d74a8682e3c5a05ebdb786c0807a9dd1f7e" Jul 28 17:14:07.853661 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.853634 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-587668d4b-dp65p"] Jul 28 17:14:07.861696 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:07.861672 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-587668d4b-dp65p"] Jul 28 17:14:08.818369 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:08.818334 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-z6qvn" event={"ID":"a5530c61-043a-404e-afad-43651d677628","Type":"ContainerStarted","Data":"d0800a26a291f2156c161ad95f6b2b6fc9f22a06d61b0e9a6ff6993e6f77533f"} Jul 28 17:14:09.723297 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:09.723257 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" path="/var/lib/kubelet/pods/baf01f69-9d9b-4bc5-ba76-47c2fe585884/volumes" Jul 28 17:14:12.589995 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.589960 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c7dd9ccf5-qxlzz"] Jul 28 17:14:12.590375 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.590326 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" containerName="registry" Jul 28 17:14:12.590375 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.590338 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" containerName="registry" Jul 28 17:14:12.590441 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.590394 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="baf01f69-9d9b-4bc5-ba76-47c2fe585884" containerName="registry" Jul 28 17:14:12.603751 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.603713 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.606321 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.606293 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7dd9ccf5-qxlzz"] Jul 28 17:14:12.606920 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.606896 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 17:14:12.607116 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.607100 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 17:14:12.607474 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.607450 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 17:14:12.607910 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.607890 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 17:14:12.608026 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.607948 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 17:14:12.608411 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.608388 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 17:14:12.608501 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.608408 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 17:14:12.608501 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.608480 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-wfgr6\"" Jul 28 17:14:12.613538 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.613516 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 17:14:12.632792 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.632764 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-serving-cert\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.632792 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.632794 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-oauth-config\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.633014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.632814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-oauth-serving-cert\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.633014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.632861 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-service-ca\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.633014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.632898 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-console-config\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.633014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.632922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhwcl\" (UniqueName: \"kubernetes.io/projected/09135161-847f-4029-91b1-78c030f60010-kube-api-access-fhwcl\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.633014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.632983 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-trusted-ca-bundle\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.733956 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.733924 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-serving-cert\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.733956 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.733960 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-oauth-config\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.734177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.733975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-oauth-serving-cert\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.734177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.734006 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-service-ca\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.734177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.734030 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-console-config\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.734177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.734052 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhwcl\" (UniqueName: \"kubernetes.io/projected/09135161-847f-4029-91b1-78c030f60010-kube-api-access-fhwcl\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.734177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.734096 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-trusted-ca-bundle\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.734927 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.734899 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-console-config\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.735065 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.734897 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-oauth-serving-cert\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.735065 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.734956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-service-ca\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.735182 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.735153 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-trusted-ca-bundle\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.736594 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.736578 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-oauth-config\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.736685 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.736668 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-serving-cert\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.745350 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.745327 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhwcl\" (UniqueName: \"kubernetes.io/projected/09135161-847f-4029-91b1-78c030f60010-kube-api-access-fhwcl\") pod \"console-7c7dd9ccf5-qxlzz\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:12.915303 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:12.915220 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:13.055558 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:13.055534 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c7dd9ccf5-qxlzz"] Jul 28 17:14:13.057507 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:14:13.057481 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09135161_847f_4029_91b1_78c030f60010.slice/crio-38f1b77e20ca5ad62b95ed9095e9b3950b68c59e201d6ed803db475b4d0c5f1a WatchSource:0}: Error finding container 38f1b77e20ca5ad62b95ed9095e9b3950b68c59e201d6ed803db475b4d0c5f1a: Status 404 returned error can't find the container with id 38f1b77e20ca5ad62b95ed9095e9b3950b68c59e201d6ed803db475b4d0c5f1a Jul 28 17:14:13.839505 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:13.839466 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7dd9ccf5-qxlzz" event={"ID":"09135161-847f-4029-91b1-78c030f60010","Type":"ContainerStarted","Data":"38f1b77e20ca5ad62b95ed9095e9b3950b68c59e201d6ed803db475b4d0c5f1a"} Jul 28 17:14:16.854901 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:16.854861 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7dd9ccf5-qxlzz" event={"ID":"09135161-847f-4029-91b1-78c030f60010","Type":"ContainerStarted","Data":"23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d"} Jul 28 17:14:16.876744 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:16.876691 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c7dd9ccf5-qxlzz" podStartSLOduration=1.4885479959999999 podStartE2EDuration="4.876674821s" totalImagesPullingTime="3.388126825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:16.875550391 +0000 UTC m=+339.634411304" watchObservedRunningTime="2026-07-28 17:14:16.876674821 +0000 UTC m=+339.635535734" Jul 28 17:14:22.916455 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:22.915686 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:22.917271 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:22.917239 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:22.925387 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:22.925367 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:23.890877 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:23.890850 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:14:24.214367 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:24.214290 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:14:24.218166 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:24.218143 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7545c9ffff-7km2p" Jul 28 17:14:27.905965 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:27.905931 2570 generic.go:356] "Generic (PLEG): container finished" podID="b1525dd4-496e-4b33-964e-db81ff558ed9" containerID="e73d5c10f3ab30b6706244523c95866bb5bba4c750736d134e898548b4151bf5" exitCode=0 Jul 28 17:14:27.906327 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:27.906004 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" event={"ID":"b1525dd4-496e-4b33-964e-db81ff558ed9","Type":"ContainerDied","Data":"e73d5c10f3ab30b6706244523c95866bb5bba4c750736d134e898548b4151bf5"} Jul 28 17:14:27.906327 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:27.906318 2570 scope.go:117] "RemoveContainer" containerID="e73d5c10f3ab30b6706244523c95866bb5bba4c750736d134e898548b4151bf5" Jul 28 17:14:28.911237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:28.911203 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh" event={"ID":"b1525dd4-496e-4b33-964e-db81ff558ed9","Type":"ContainerStarted","Data":"ff5b97bd9d167106397c7fd2f80ef9237c194753a7c13b92c1741cadfaa9fca7"} Jul 28 17:14:32.928007 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:32.927972 2570 generic.go:356] "Generic (PLEG): container finished" podID="46326b21-01e9-4e63-85c8-516ea6d5d862" containerID="be249719f4fc71d70e1b8b9cbe9fe5b50b0221f0fdf7e6addcc40cf1ff8301e4" exitCode=0 Jul 28 17:14:32.928398 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:32.928019 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" event={"ID":"46326b21-01e9-4e63-85c8-516ea6d5d862","Type":"ContainerDied","Data":"be249719f4fc71d70e1b8b9cbe9fe5b50b0221f0fdf7e6addcc40cf1ff8301e4"} Jul 28 17:14:32.928398 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:32.928328 2570 scope.go:117] "RemoveContainer" containerID="be249719f4fc71d70e1b8b9cbe9fe5b50b0221f0fdf7e6addcc40cf1ff8301e4" Jul 28 17:14:33.934066 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:33.934033 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-ddhsz" event={"ID":"46326b21-01e9-4e63-85c8-516ea6d5d862","Type":"ContainerStarted","Data":"9d182b0fe08dd8185adb2e451fe463081a9a0aea95cd758238a785972c778282"} Jul 28 17:14:51.673704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:51.673661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:14:51.675896 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:51.675878 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b00417f8-bed7-4fcb-b3ec-f912fc8dfb91-metrics-certs\") pod \"network-metrics-daemon-plwl2\" (UID: \"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91\") " pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:14:51.722650 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:51.722620 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-99hnw\"" Jul 28 17:14:51.729882 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:51.729864 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-plwl2" Jul 28 17:14:51.858636 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:51.858605 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-plwl2"] Jul 28 17:14:51.862217 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:14:51.862193 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00417f8_bed7_4fcb_b3ec_f912fc8dfb91.slice/crio-42a7f29a78807c45ca6ac0b1dc57ef439802f4ff0c5d03ba03905f0a5fbf9eba WatchSource:0}: Error finding container 42a7f29a78807c45ca6ac0b1dc57ef439802f4ff0c5d03ba03905f0a5fbf9eba: Status 404 returned error can't find the container with id 42a7f29a78807c45ca6ac0b1dc57ef439802f4ff0c5d03ba03905f0a5fbf9eba Jul 28 17:14:52.006860 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:52.006755 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-plwl2" event={"ID":"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91","Type":"ContainerStarted","Data":"42a7f29a78807c45ca6ac0b1dc57ef439802f4ff0c5d03ba03905f0a5fbf9eba"} Jul 28 17:14:53.013122 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:53.013034 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-plwl2" event={"ID":"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91","Type":"ContainerStarted","Data":"cf7851de9951cfa64076875546c1e2b313e007738a204ad4db6d703d4f8f18c2"} Jul 28 17:14:53.013122 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:53.013083 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-plwl2" event={"ID":"b00417f8-bed7-4fcb-b3ec-f912fc8dfb91","Type":"ContainerStarted","Data":"41eca9c97e83cc4822f6767faaa898d95879592e07fb25414bf4e889284d396e"} Jul 28 17:14:53.033392 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:53.033293 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-plwl2" podStartSLOduration=374.16330158 podStartE2EDuration="6m15.033277485s" totalImagesPullingTime="869.975905ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:53.031165712 +0000 UTC m=+375.790026626" watchObservedRunningTime="2026-07-28 17:14:53.033277485 +0000 UTC m=+375.792138398" Jul 28 17:14:55.855044 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.855010 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59b69bb7d8-nht6n"] Jul 28 17:14:55.858269 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.858251 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:55.881620 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.881595 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59b69bb7d8-nht6n"] Jul 28 17:14:55.910403 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.910377 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-config\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:55.910550 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.910407 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-oauth-serving-cert\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:55.910550 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.910499 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtbmg\" (UniqueName: \"kubernetes.io/projected/182a260d-ab01-47b7-b963-e3f4b96ebed1-kube-api-access-wtbmg\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:55.910550 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.910527 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-serving-cert\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:55.910550 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.910545 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-service-ca\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:55.910705 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.910565 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-trusted-ca-bundle\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:55.910705 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:55.910581 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-oauth-config\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.011666 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.011626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtbmg\" (UniqueName: \"kubernetes.io/projected/182a260d-ab01-47b7-b963-e3f4b96ebed1-kube-api-access-wtbmg\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.011666 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.011667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-serving-cert\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.011955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.011690 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-service-ca\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.011955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.011716 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-trusted-ca-bundle\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.011955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.011737 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-oauth-config\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.011955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.011776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-config\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.011955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.011801 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-oauth-serving-cert\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.012589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.012564 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-service-ca\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.012589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.012570 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-oauth-serving-cert\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.012725 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.012664 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-config\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.012876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.012854 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-trusted-ca-bundle\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.014243 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.014226 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-oauth-config\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.014423 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.014404 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-serving-cert\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.021463 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.021440 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtbmg\" (UniqueName: \"kubernetes.io/projected/182a260d-ab01-47b7-b963-e3f4b96ebed1-kube-api-access-wtbmg\") pod \"console-59b69bb7d8-nht6n\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.167618 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.167515 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:14:56.299333 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:56.299311 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59b69bb7d8-nht6n"] Jul 28 17:14:56.301339 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:14:56.301306 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182a260d_ab01_47b7_b963_e3f4b96ebed1.slice/crio-1482eee1843f7b957897f28843c255ecf6e35ccd13a27c2bb7f658517a852b63 WatchSource:0}: Error finding container 1482eee1843f7b957897f28843c255ecf6e35ccd13a27c2bb7f658517a852b63: Status 404 returned error can't find the container with id 1482eee1843f7b957897f28843c255ecf6e35ccd13a27c2bb7f658517a852b63 Jul 28 17:14:57.029923 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:57.029889 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b69bb7d8-nht6n" event={"ID":"182a260d-ab01-47b7-b963-e3f4b96ebed1","Type":"ContainerStarted","Data":"99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4"} Jul 28 17:14:57.030280 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:57.029936 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b69bb7d8-nht6n" event={"ID":"182a260d-ab01-47b7-b963-e3f4b96ebed1","Type":"ContainerStarted","Data":"1482eee1843f7b957897f28843c255ecf6e35ccd13a27c2bb7f658517a852b63"} Jul 28 17:14:57.051205 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:57.051152 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59b69bb7d8-nht6n" podStartSLOduration=2.051136878 podStartE2EDuration="2.051136878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:57.050291335 +0000 UTC m=+379.809152249" watchObservedRunningTime="2026-07-28 17:14:57.051136878 +0000 UTC m=+379.809997787" Jul 28 17:14:59.956770 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:59.956734 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:14:59.957189 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:59.957035 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="alertmanager" containerID="cri-o://dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e" gracePeriod=120 Jul 28 17:14:59.957189 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:59.957109 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-web" containerID="cri-o://ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2" gracePeriod=120 Jul 28 17:14:59.957189 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:59.957139 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="prom-label-proxy" containerID="cri-o://f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51" gracePeriod=120 Jul 28 17:14:59.957189 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:59.957156 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="config-reloader" containerID="cri-o://5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109" gracePeriod=120 Jul 28 17:14:59.957189 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:59.957163 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy" containerID="cri-o://c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e" gracePeriod=120 Jul 28 17:14:59.957189 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:14:59.957111 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-metric" containerID="cri-o://1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0" gracePeriod=120 Jul 28 17:15:01.053149 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053115 2570 generic.go:356] "Generic (PLEG): container finished" podID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerID="f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51" exitCode=0 Jul 28 17:15:01.053149 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053143 2570 generic.go:356] "Generic (PLEG): container finished" podID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerID="1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0" exitCode=0 Jul 28 17:15:01.053149 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053150 2570 generic.go:356] "Generic (PLEG): container finished" podID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerID="c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e" exitCode=0 Jul 28 17:15:01.053149 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053155 2570 generic.go:356] "Generic (PLEG): container finished" podID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerID="5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109" exitCode=0 Jul 28 17:15:01.053599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053162 2570 generic.go:356] "Generic (PLEG): container finished" podID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerID="dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e" exitCode=0 Jul 28 17:15:01.053599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053144 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51"} Jul 28 17:15:01.053599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053300 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0"} Jul 28 17:15:01.053599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e"} Jul 28 17:15:01.053599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053364 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109"} Jul 28 17:15:01.053599 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.053387 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e"} Jul 28 17:15:01.217193 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.217167 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:01.254430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254402 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.254589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254437 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-cluster-tls-config\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.254589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254459 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-tls-assets\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.254589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254503 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-out\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.254589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254519 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.254589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254552 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-metrics-client-ca\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.254589 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254571 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-web-config\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.255049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254592 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.255049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254632 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-main-tls\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.255049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254691 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-main-db\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.255049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254717 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt8x8\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-kube-api-access-qt8x8\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.255049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254754 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-volume\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.255049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254796 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-web\") pod \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\" (UID: \"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2\") " Jul 28 17:15:01.255049 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.254965 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:15:01.255407 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.255114 2570 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-metrics-client-ca\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.255407 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.255323 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:15:01.255600 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.255553 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:15:01.257721 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.257681 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:01.258305 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.258267 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:15:01.258499 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.258455 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:01.258499 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.258476 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:01.258734 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.258708 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:01.258734 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.258714 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-out" (OuterVolumeSpecName: "config-out") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:15:01.258892 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.258860 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-volume" (OuterVolumeSpecName: "config-volume") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:01.259153 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.259131 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-kube-api-access-qt8x8" (OuterVolumeSpecName: "kube-api-access-qt8x8") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "kube-api-access-qt8x8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:15:01.262151 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.262130 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:01.269540 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.269519 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-web-config" (OuterVolumeSpecName: "web-config") pod "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" (UID: "b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:01.356522 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356483 2570 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-main-db\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356522 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356515 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt8x8\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-kube-api-access-qt8x8\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356522 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356526 2570 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-volume\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356536 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356547 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356557 2570 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-cluster-tls-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356565 2570 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-tls-assets\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356573 2570 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-config-out\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356580 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356588 2570 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-web-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356596 2570 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:01.356768 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:01.356604 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2-secret-alertmanager-main-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:02.061169 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.061083 2570 generic.go:356] "Generic (PLEG): container finished" podID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerID="ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2" exitCode=0 Jul 28 17:15:02.061524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.061164 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2"} Jul 28 17:15:02.061524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.061231 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2","Type":"ContainerDied","Data":"f51e027841c3b3f73d3791e0d22b9e7da1656f0141f0c0098900d083333b2be0"} Jul 28 17:15:02.061524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.061172 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.061524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.061264 2570 scope.go:117] "RemoveContainer" containerID="f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51" Jul 28 17:15:02.069162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.069144 2570 scope.go:117] "RemoveContainer" containerID="1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0" Jul 28 17:15:02.076771 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.076660 2570 scope.go:117] "RemoveContainer" containerID="c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e" Jul 28 17:15:02.084192 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.084167 2570 scope.go:117] "RemoveContainer" containerID="ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2" Jul 28 17:15:02.091299 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.091278 2570 scope.go:117] "RemoveContainer" containerID="5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109" Jul 28 17:15:02.099053 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.099026 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:15:02.099984 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.099968 2570 scope.go:117] "RemoveContainer" containerID="dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e" Jul 28 17:15:02.104444 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.104420 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:15:02.108470 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.108445 2570 scope.go:117] "RemoveContainer" containerID="bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db" Jul 28 17:15:02.115903 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.115887 2570 scope.go:117] "RemoveContainer" containerID="f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51" Jul 28 17:15:02.116203 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:02.116178 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51\": container with ID starting with f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51 not found: ID does not exist" containerID="f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51" Jul 28 17:15:02.116255 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116213 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51"} err="failed to get container status \"f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51\": rpc error: code = NotFound desc = could not find container \"f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51\": container with ID starting with f7046e759d48a68a50adf85f8e24daab8b9220b471c5b48c886b6a3a6602de51 not found: ID does not exist" Jul 28 17:15:02.116255 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116246 2570 scope.go:117] "RemoveContainer" containerID="1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0" Jul 28 17:15:02.116488 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:02.116474 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0\": container with ID starting with 1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0 not found: ID does not exist" containerID="1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0" Jul 28 17:15:02.116531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116493 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0"} err="failed to get container status \"1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0\": rpc error: code = NotFound desc = could not find container \"1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0\": container with ID starting with 1dd05f2d42c98e5fe6b6f475732de290a9fc18fa853fb0e21f2066050d0855f0 not found: ID does not exist" Jul 28 17:15:02.116531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116507 2570 scope.go:117] "RemoveContainer" containerID="c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e" Jul 28 17:15:02.116717 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:02.116700 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e\": container with ID starting with c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e not found: ID does not exist" containerID="c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e" Jul 28 17:15:02.116757 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116722 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e"} err="failed to get container status \"c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e\": rpc error: code = NotFound desc = could not find container \"c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e\": container with ID starting with c893f70013286173888a6363e9adf8130ad747b572c7b1b6855bfc8b9b405b4e not found: ID does not exist" Jul 28 17:15:02.116757 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116735 2570 scope.go:117] "RemoveContainer" containerID="ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2" Jul 28 17:15:02.116931 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:02.116913 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2\": container with ID starting with ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2 not found: ID does not exist" containerID="ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2" Jul 28 17:15:02.116984 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116937 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2"} err="failed to get container status \"ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2\": rpc error: code = NotFound desc = could not find container \"ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2\": container with ID starting with ecba25df7088d8341c5edc5345a182ada20649aab7e608ae6d2ed4ef7b8929b2 not found: ID does not exist" Jul 28 17:15:02.116984 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.116954 2570 scope.go:117] "RemoveContainer" containerID="5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109" Jul 28 17:15:02.117173 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:02.117158 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109\": container with ID starting with 5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109 not found: ID does not exist" containerID="5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109" Jul 28 17:15:02.117217 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.117176 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109"} err="failed to get container status \"5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109\": rpc error: code = NotFound desc = could not find container \"5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109\": container with ID starting with 5857fd0a1106b5eefa9bb26bf45baace6ffdb303549b33e7d219c9d2e0fbd109 not found: ID does not exist" Jul 28 17:15:02.117217 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.117189 2570 scope.go:117] "RemoveContainer" containerID="dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e" Jul 28 17:15:02.117413 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:02.117396 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e\": container with ID starting with dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e not found: ID does not exist" containerID="dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e" Jul 28 17:15:02.117453 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.117417 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e"} err="failed to get container status \"dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e\": rpc error: code = NotFound desc = could not find container \"dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e\": container with ID starting with dd171c0d55567527a362ea3f7629303fed43d326ab79bd39e7517645d285b66e not found: ID does not exist" Jul 28 17:15:02.117453 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.117431 2570 scope.go:117] "RemoveContainer" containerID="bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db" Jul 28 17:15:02.117658 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:02.117642 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db\": container with ID starting with bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db not found: ID does not exist" containerID="bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db" Jul 28 17:15:02.117709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.117660 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db"} err="failed to get container status \"bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db\": rpc error: code = NotFound desc = could not find container \"bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db\": container with ID starting with bcad859dc7bdb0c22bcd4e4fce0492b5a2cde0fb6ffcc5d9559aa72b113955db not found: ID does not exist" Jul 28 17:15:02.147104 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147061 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:15:02.147482 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147465 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="config-reloader" Jul 28 17:15:02.147535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147485 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="config-reloader" Jul 28 17:15:02.147535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147496 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="init-config-reloader" Jul 28 17:15:02.147535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147502 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="init-config-reloader" Jul 28 17:15:02.147535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147509 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-metric" Jul 28 17:15:02.147535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147514 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-metric" Jul 28 17:15:02.147535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147523 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="prom-label-proxy" Jul 28 17:15:02.147535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147528 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="prom-label-proxy" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147539 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-web" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147544 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-web" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147555 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="alertmanager" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147560 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="alertmanager" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147565 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147570 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147615 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="alertmanager" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147622 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-web" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147630 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="config-reloader" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147636 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147641 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="kube-rbac-proxy-metric" Jul 28 17:15:02.147732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.147646 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" containerName="prom-label-proxy" Jul 28 17:15:02.152607 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.152590 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.159355 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.159324 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 17:15:02.159497 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.159360 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 17:15:02.159497 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.159394 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 17:15:02.159660 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.159645 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 17:15:02.159773 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.159749 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-pgprt\"" Jul 28 17:15:02.159847 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.159792 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 17:15:02.159901 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.159863 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 17:15:02.161440 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.161425 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 17:15:02.164544 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.164529 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 17:15:02.171683 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.171659 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 17:15:02.180014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.179979 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:15:02.265375 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265341 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7673ac2-4d8f-431b-873b-49a606675b24-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265375 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265377 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7673ac2-4d8f-431b-873b-49a606675b24-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265405 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265443 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7673ac2-4d8f-431b-873b-49a606675b24-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265464 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265490 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265506 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265897 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265606 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-web-config\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265897 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265650 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7673ac2-4d8f-431b-873b-49a606675b24-config-out\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265897 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265732 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265897 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265761 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265897 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265790 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7673ac2-4d8f-431b-873b-49a606675b24-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.265897 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.265823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brbwq\" (UniqueName: \"kubernetes.io/projected/e7673ac2-4d8f-431b-873b-49a606675b24-kube-api-access-brbwq\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.366877 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.366808 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.366877 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.366883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.366918 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-web-config\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.366938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7673ac2-4d8f-431b-873b-49a606675b24-config-out\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.366967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.366983 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.366999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7673ac2-4d8f-431b-873b-49a606675b24-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.367015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brbwq\" (UniqueName: \"kubernetes.io/projected/e7673ac2-4d8f-431b-873b-49a606675b24-kube-api-access-brbwq\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.367033 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7673ac2-4d8f-431b-873b-49a606675b24-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.367051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7673ac2-4d8f-431b-873b-49a606675b24-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367085 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.367080 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367344 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.367096 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7673ac2-4d8f-431b-873b-49a606675b24-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367344 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.367116 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.367919 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.367889 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7673ac2-4d8f-431b-873b-49a606675b24-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.368774 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.368424 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7673ac2-4d8f-431b-873b-49a606675b24-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.370234 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.370207 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7673ac2-4d8f-431b-873b-49a606675b24-config-out\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.370341 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.370318 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.370410 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.370352 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.370466 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.370421 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-web-config\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.370546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.370522 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.370659 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.370551 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7673ac2-4d8f-431b-873b-49a606675b24-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.370891 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.370868 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7673ac2-4d8f-431b-873b-49a606675b24-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.371123 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.371103 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.371262 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.371245 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.372257 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.372241 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7673ac2-4d8f-431b-873b-49a606675b24-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.380534 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.380512 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brbwq\" (UniqueName: \"kubernetes.io/projected/e7673ac2-4d8f-431b-873b-49a606675b24-kube-api-access-brbwq\") pod \"alertmanager-main-0\" (UID: \"e7673ac2-4d8f-431b-873b-49a606675b24\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.463250 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.463198 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:15:02.625055 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:02.624908 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:15:02.627807 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:15:02.627780 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7673ac2_4d8f_431b_873b_49a606675b24.slice/crio-74e22b6af126ab6dab6b436cec1d65a60411b33f1bbf54eb43f811614bc5d4d9 WatchSource:0}: Error finding container 74e22b6af126ab6dab6b436cec1d65a60411b33f1bbf54eb43f811614bc5d4d9: Status 404 returned error can't find the container with id 74e22b6af126ab6dab6b436cec1d65a60411b33f1bbf54eb43f811614bc5d4d9 Jul 28 17:15:03.068751 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:03.068661 2570 generic.go:356] "Generic (PLEG): container finished" podID="e7673ac2-4d8f-431b-873b-49a606675b24" containerID="0e8ae4858f534322d304a2e11f3f1c4b34ec11f98006110e8c4cb37dec05e0f1" exitCode=0 Jul 28 17:15:03.068751 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:03.068724 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerDied","Data":"0e8ae4858f534322d304a2e11f3f1c4b34ec11f98006110e8c4cb37dec05e0f1"} Jul 28 17:15:03.069142 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:03.068780 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerStarted","Data":"74e22b6af126ab6dab6b436cec1d65a60411b33f1bbf54eb43f811614bc5d4d9"} Jul 28 17:15:03.723172 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:03.723140 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2" path="/var/lib/kubelet/pods/b6f9bdb2-5a96-41e3-bc9a-cea72e6e1bf2/volumes" Jul 28 17:15:04.078216 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:04.078120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerStarted","Data":"364475caf6c877e0c46b9ff8fc68f50fd14713a9e3075c672aa63dae577ef583"} Jul 28 17:15:04.078216 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:04.078206 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerStarted","Data":"3c784871c50da9cc72485dbca15546ff32dc462db43cb6a9c837b1587a84715b"} Jul 28 17:15:04.078730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:04.078254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerStarted","Data":"a0c86636d8a21e1a23d46590277a3cfed72e6c85b87d293c1a4ef5dd9cfc89ea"} Jul 28 17:15:04.078730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:04.078290 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerStarted","Data":"9ab07c6762925ec32f2195ed96a2b6ec4082be02401a524b681ec50f40544226"} Jul 28 17:15:04.078730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:04.078336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerStarted","Data":"71c008b5844e55e642e2c07a41d850184aae0d285ff6a8556a86b85f5b44de41"} Jul 28 17:15:04.078730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:04.078379 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7673ac2-4d8f-431b-873b-49a606675b24","Type":"ContainerStarted","Data":"cedb02813157d2ac98b2a4e686704585ea3525642cde941ab2e0b6242bbdae7b"} Jul 28 17:15:04.121242 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:04.121184 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.105348767 podStartE2EDuration="2.12116296s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.814193ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:15:04.120873207 +0000 UTC m=+386.879734121" watchObservedRunningTime="2026-07-28 17:15:04.12116296 +0000 UTC m=+386.880023873" Jul 28 17:15:06.167905 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:06.167866 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:15:06.168386 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:06.167935 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:15:06.172549 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:06.172523 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:15:07.095713 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:07.095684 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:15:07.150089 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:07.150054 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c7dd9ccf5-qxlzz"] Jul 28 17:15:32.169361 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.169317 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c7dd9ccf5-qxlzz" podUID="09135161-847f-4029-91b1-78c030f60010" containerName="console" containerID="cri-o://23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d" gracePeriod=15 Jul 28 17:15:32.417616 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.417591 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:15:32.537913 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.537783 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-oauth-config\") pod \"09135161-847f-4029-91b1-78c030f60010\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " Jul 28 17:15:32.537913 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.537893 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-serving-cert\") pod \"09135161-847f-4029-91b1-78c030f60010\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " Jul 28 17:15:32.538141 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.537942 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-service-ca\") pod \"09135161-847f-4029-91b1-78c030f60010\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " Jul 28 17:15:32.538141 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.537969 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-trusted-ca-bundle\") pod \"09135161-847f-4029-91b1-78c030f60010\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " Jul 28 17:15:32.538141 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.538021 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-oauth-serving-cert\") pod \"09135161-847f-4029-91b1-78c030f60010\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " Jul 28 17:15:32.538141 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.538046 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhwcl\" (UniqueName: \"kubernetes.io/projected/09135161-847f-4029-91b1-78c030f60010-kube-api-access-fhwcl\") pod \"09135161-847f-4029-91b1-78c030f60010\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " Jul 28 17:15:32.538141 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.538069 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-console-config\") pod \"09135161-847f-4029-91b1-78c030f60010\" (UID: \"09135161-847f-4029-91b1-78c030f60010\") " Jul 28 17:15:32.538619 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.538584 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "09135161-847f-4029-91b1-78c030f60010" (UID: "09135161-847f-4029-91b1-78c030f60010"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:15:32.538709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.538626 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "09135161-847f-4029-91b1-78c030f60010" (UID: "09135161-847f-4029-91b1-78c030f60010"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:15:32.538749 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.538707 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-service-ca" (OuterVolumeSpecName: "service-ca") pod "09135161-847f-4029-91b1-78c030f60010" (UID: "09135161-847f-4029-91b1-78c030f60010"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:15:32.538749 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.538725 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-console-config" (OuterVolumeSpecName: "console-config") pod "09135161-847f-4029-91b1-78c030f60010" (UID: "09135161-847f-4029-91b1-78c030f60010"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:15:32.540224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.540197 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "09135161-847f-4029-91b1-78c030f60010" (UID: "09135161-847f-4029-91b1-78c030f60010"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:32.540323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.540220 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "09135161-847f-4029-91b1-78c030f60010" (UID: "09135161-847f-4029-91b1-78c030f60010"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:15:32.540323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.540283 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09135161-847f-4029-91b1-78c030f60010-kube-api-access-fhwcl" (OuterVolumeSpecName: "kube-api-access-fhwcl") pod "09135161-847f-4029-91b1-78c030f60010" (UID: "09135161-847f-4029-91b1-78c030f60010"). InnerVolumeSpecName "kube-api-access-fhwcl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:15:32.639266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.639219 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:32.639266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.639261 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-service-ca\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:32.639266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.639271 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-trusted-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:32.639266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.639280 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-oauth-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:32.639266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.639288 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhwcl\" (UniqueName: \"kubernetes.io/projected/09135161-847f-4029-91b1-78c030f60010-kube-api-access-fhwcl\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:32.639569 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.639297 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09135161-847f-4029-91b1-78c030f60010-console-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:32.639569 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:32.639304 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09135161-847f-4029-91b1-78c030f60010-console-oauth-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:15:33.207523 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.207484 2570 generic.go:356] "Generic (PLEG): container finished" podID="09135161-847f-4029-91b1-78c030f60010" containerID="23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d" exitCode=0 Jul 28 17:15:33.207950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.207532 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c7dd9ccf5-qxlzz" Jul 28 17:15:33.207950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.207564 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7dd9ccf5-qxlzz" event={"ID":"09135161-847f-4029-91b1-78c030f60010","Type":"ContainerDied","Data":"23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d"} Jul 28 17:15:33.207950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.207618 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c7dd9ccf5-qxlzz" event={"ID":"09135161-847f-4029-91b1-78c030f60010","Type":"ContainerDied","Data":"38f1b77e20ca5ad62b95ed9095e9b3950b68c59e201d6ed803db475b4d0c5f1a"} Jul 28 17:15:33.207950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.207640 2570 scope.go:117] "RemoveContainer" containerID="23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d" Jul 28 17:15:33.216607 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.216586 2570 scope.go:117] "RemoveContainer" containerID="23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d" Jul 28 17:15:33.216876 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:15:33.216855 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d\": container with ID starting with 23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d not found: ID does not exist" containerID="23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d" Jul 28 17:15:33.216955 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.216882 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d"} err="failed to get container status \"23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d\": rpc error: code = NotFound desc = could not find container \"23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d\": container with ID starting with 23c9a69ca1d315d6d3e161e09f178bd4f5dacf720039bf883ac87e2e7972c83d not found: ID does not exist" Jul 28 17:15:33.230656 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.230632 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c7dd9ccf5-qxlzz"] Jul 28 17:15:33.237979 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.237955 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c7dd9ccf5-qxlzz"] Jul 28 17:15:33.723025 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:15:33.722991 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09135161-847f-4029-91b1-78c030f60010" path="/var/lib/kubelet/pods/09135161-847f-4029-91b1-78c030f60010/volumes" Jul 28 17:17:18.537334 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.537298 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt"] Jul 28 17:17:18.537821 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.537658 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09135161-847f-4029-91b1-78c030f60010" containerName="console" Jul 28 17:17:18.537821 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.537670 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="09135161-847f-4029-91b1-78c030f60010" containerName="console" Jul 28 17:17:18.537821 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.537734 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="09135161-847f-4029-91b1-78c030f60010" containerName="console" Jul 28 17:17:18.540644 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.540628 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:17:18.544018 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.543999 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\"" Jul 28 17:17:18.545119 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.545101 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\"" Jul 28 17:17:18.545216 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.545103 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-rh8nj\"" Jul 28 17:17:18.562463 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.562435 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt"] Jul 28 17:17:18.627251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.627211 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh84p\" (UniqueName: \"kubernetes.io/projected/096131fa-3f5c-4d3e-9738-533c1b0260b0-kube-api-access-nh84p\") pod \"feast-operator-controller-manager-7b7db985fb-zhntt\" (UID: \"096131fa-3f5c-4d3e-9738-533c1b0260b0\") " pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:17:18.728358 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.728313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh84p\" (UniqueName: \"kubernetes.io/projected/096131fa-3f5c-4d3e-9738-533c1b0260b0-kube-api-access-nh84p\") pod \"feast-operator-controller-manager-7b7db985fb-zhntt\" (UID: \"096131fa-3f5c-4d3e-9738-533c1b0260b0\") " pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:17:18.751210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.751187 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh84p\" (UniqueName: \"kubernetes.io/projected/096131fa-3f5c-4d3e-9738-533c1b0260b0-kube-api-access-nh84p\") pod \"feast-operator-controller-manager-7b7db985fb-zhntt\" (UID: \"096131fa-3f5c-4d3e-9738-533c1b0260b0\") " pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:17:18.851034 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.851000 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:17:18.994665 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:18.994621 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt"] Jul 28 17:17:18.999707 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:17:18.999677 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod096131fa_3f5c_4d3e_9738_533c1b0260b0.slice/crio-15656deeedadd423dccebdb8e78c1f368e63ecb646d9edd86f2dbf0c30609da8 WatchSource:0}: Error finding container 15656deeedadd423dccebdb8e78c1f368e63ecb646d9edd86f2dbf0c30609da8: Status 404 returned error can't find the container with id 15656deeedadd423dccebdb8e78c1f368e63ecb646d9edd86f2dbf0c30609da8 Jul 28 17:17:19.652468 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:19.652427 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" event={"ID":"096131fa-3f5c-4d3e-9738-533c1b0260b0","Type":"ContainerStarted","Data":"15656deeedadd423dccebdb8e78c1f368e63ecb646d9edd86f2dbf0c30609da8"} Jul 28 17:17:21.663320 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:21.663281 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" event={"ID":"096131fa-3f5c-4d3e-9738-533c1b0260b0","Type":"ContainerStarted","Data":"6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31"} Jul 28 17:17:21.663749 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:21.663411 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:17:21.686411 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:21.686362 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" podStartSLOduration=1.366162622 podStartE2EDuration="3.686347251s" totalImagesPullingTime="2.320184629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:21.685199979 +0000 UTC m=+524.444060894" watchObservedRunningTime="2026-07-28 17:17:21.686347251 +0000 UTC m=+524.445208165" Jul 28 17:17:32.670577 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:32.670491 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:17:58.538523 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.538486 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q"] Jul 28 17:17:58.547324 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.547302 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.551633 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.551606 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 28 17:17:58.551811 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.551797 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 28 17:17:58.552246 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.552225 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-jrssw\"" Jul 28 17:17:58.552246 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.552236 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 28 17:17:58.552418 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.552236 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 28 17:17:58.552418 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.552226 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 28 17:17:58.553027 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.553008 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 28 17:17:58.553114 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.553044 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 28 17:17:58.571055 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.571028 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q"] Jul 28 17:17:58.654054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.654024 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-registry-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.654054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.654057 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/774ef898-20b6-4061-a731-4e9cc83b5200-feast-data\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.654251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.654137 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6282\" (UniqueName: \"kubernetes.io/projected/774ef898-20b6-4061-a731-4e9cc83b5200-kube-api-access-f6282\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.654251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.654185 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-online-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.654251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.654206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-ui-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.654251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.654235 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-offline-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.755516 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.755463 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-registry-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.755516 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.755513 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/774ef898-20b6-4061-a731-4e9cc83b5200-feast-data\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.755767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.755555 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6282\" (UniqueName: \"kubernetes.io/projected/774ef898-20b6-4061-a731-4e9cc83b5200-kube-api-access-f6282\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.755767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.755583 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-online-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.755767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.755600 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-ui-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.755767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.755681 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-offline-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.756115 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.756095 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/774ef898-20b6-4061-a731-4e9cc83b5200-feast-data\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.758068 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.758043 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-online-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.758234 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.758215 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-registry-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.758280 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.758222 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-offline-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.758318 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.758285 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-ui-tls\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.770210 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.770183 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6282\" (UniqueName: \"kubernetes.io/projected/774ef898-20b6-4061-a731-4e9cc83b5200-kube-api-access-f6282\") pod \"feast-simple-feast-setup-56cb884f74-6tj4q\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:58.867741 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:58.867702 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:17:59.045397 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:59.045359 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q"] Jul 28 17:17:59.049983 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:17:59.049954 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod774ef898_20b6_4061_a731_4e9cc83b5200.slice/crio-726aa7b4d01512d587fea818cfef5e99a18d07acdfbb0a8e06a68aa1d9057e6a WatchSource:0}: Error finding container 726aa7b4d01512d587fea818cfef5e99a18d07acdfbb0a8e06a68aa1d9057e6a: Status 404 returned error can't find the container with id 726aa7b4d01512d587fea818cfef5e99a18d07acdfbb0a8e06a68aa1d9057e6a Jul 28 17:17:59.825657 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:17:59.825606 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerStarted","Data":"726aa7b4d01512d587fea818cfef5e99a18d07acdfbb0a8e06a68aa1d9057e6a"} Jul 28 17:18:28.974173 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:28.974071 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerStarted","Data":"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f"} Jul 28 17:18:33.998787 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:33.998702 2570 generic.go:356] "Generic (PLEG): container finished" podID="774ef898-20b6-4061-a731-4e9cc83b5200" containerID="da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f" exitCode=0 Jul 28 17:18:33.998787 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:33.998764 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerDied","Data":"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f"} Jul 28 17:18:35.005082 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:35.005048 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerStarted","Data":"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e"} Jul 28 17:18:40.030442 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:40.030407 2570 generic.go:356] "Generic (PLEG): container finished" podID="774ef898-20b6-4061-a731-4e9cc83b5200" containerID="4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e" exitCode=0 Jul 28 17:18:40.030442 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:40.030445 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerDied","Data":"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e"} Jul 28 17:18:40.041952 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:40.041929 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 28 17:18:41.042203 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:41.042165 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerStarted","Data":"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917"} Jul 28 17:18:41.042684 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:41.042229 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerStarted","Data":"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23"} Jul 28 17:18:41.042684 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:41.042250 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerStarted","Data":"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d"} Jul 28 17:18:41.042684 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:41.042269 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerStarted","Data":"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d"} Jul 28 17:18:41.075416 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:41.075333 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podStartSLOduration=13.403984951 podStartE2EDuration="43.075310707s" totalImagesPullingTime="29.671325756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:41.071481355 +0000 UTC m=+603.830342269" watchObservedRunningTime="2026-07-28 17:18:41.075310707 +0000 UTC m=+603.834171626" Jul 28 17:18:43.868353 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.868315 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:43.868972 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.868945 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:43.869106 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.868979 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:43.869106 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.868999 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:43.870818 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.870774 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 28 17:18:43.870818 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.870796 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.29:6571: connect: connection refused" Jul 28 17:18:43.871016 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.870848 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 28 17:18:43.871016 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:43.870890 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.29:8816: connect: connection refused" Jul 28 17:18:46.868728 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.868684 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.869473 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.869448 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.869649 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.869637 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.870084 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.870062 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.870209 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.870192 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.870287 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.870221 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.870334 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.870288 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.870470 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.870453 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.871979 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.871961 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:46.874490 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:46.874475 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:47.079127 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:47.079096 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:18:47.082436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:18:47.082411 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:19:33.947938 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:33.947901 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59b69bb7d8-nht6n"] Jul 28 17:19:49.759752 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:49.759710 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q"] Jul 28 17:19:49.760159 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:49.760044 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="registry" containerID="cri-o://8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d" gracePeriod=30 Jul 28 17:19:49.760159 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:49.760092 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" containerID="cri-o://1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917" gracePeriod=30 Jul 28 17:19:49.760286 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:49.760140 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" containerID="cri-o://3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d" gracePeriod=30 Jul 28 17:19:49.760286 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:49.760098 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="offline" containerID="cri-o://bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23" gracePeriod=30 Jul 28 17:19:50.028208 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.028127 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct"] Jul 28 17:19:50.031458 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.031441 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.034762 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.034736 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-lx8v4\"" Jul 28 17:19:50.058282 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.058252 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct"] Jul 28 17:19:50.163488 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.163448 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.163488 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.163492 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-registry-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.163766 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.163527 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-feast-data\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.163766 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.163553 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.163766 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.163602 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5svb\" (UniqueName: \"kubernetes.io/projected/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-kube-api-access-v5svb\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.163766 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.163636 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-online-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.264744 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.264699 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.264950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.264760 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-registry-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.264950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.264796 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-feast-data\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.264950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.264818 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.264950 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:19:50.264882 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found Jul 28 17:19:50.264950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.264907 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5svb\" (UniqueName: \"kubernetes.io/projected/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-kube-api-access-v5svb\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.264950 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:19:50.264952 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls podName:4e17ffd4-d4b4-4718-a48c-03d0f16c23ec nodeName:}" failed. No retries permitted until 2026-07-28 17:19:50.764928991 +0000 UTC m=+673.523789883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls") pod "feast-simple-feast-setup-56cb884f74-lg2ct" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec") : secret "feast-simple-feast-setup-offline-tls" not found Jul 28 17:19:50.265449 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.264970 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-online-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.265449 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:19:50.265303 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found Jul 28 17:19:50.265449 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:19:50.265365 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls podName:4e17ffd4-d4b4-4718-a48c-03d0f16c23ec nodeName:}" failed. No retries permitted until 2026-07-28 17:19:50.76534644 +0000 UTC m=+673.524207353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls") pod "feast-simple-feast-setup-56cb884f74-lg2ct" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec") : secret "feast-simple-feast-setup-ui-tls" not found Jul 28 17:19:50.265635 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.265508 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-feast-data\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.267425 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.267401 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-online-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.267550 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.267528 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-registry-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.297047 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.296964 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5svb\" (UniqueName: \"kubernetes.io/projected/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-kube-api-access-v5svb\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.770457 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.770421 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.770856 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.770472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.772864 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.772818 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.772973 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.772883 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls\") pod \"feast-simple-feast-setup-56cb884f74-lg2ct\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:50.941635 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:50.941595 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:19:51.083717 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:51.083687 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct"] Jul 28 17:19:51.084522 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:19:51.084502 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e17ffd4_d4b4_4718_a48c_03d0f16c23ec.slice/crio-f90afa31e2f172b0c6cb622c851886ca677f4fce8dce4087287adde09d34e82e WatchSource:0}: Error finding container f90afa31e2f172b0c6cb622c851886ca677f4fce8dce4087287adde09d34e82e: Status 404 returned error can't find the container with id f90afa31e2f172b0c6cb622c851886ca677f4fce8dce4087287adde09d34e82e Jul 28 17:19:51.371697 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:51.371663 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerStarted","Data":"e0c3c774b89a3e6a73af9eea9a6b90e970af400ce25e64ec31fab634a2fa1b0c"} Jul 28 17:19:51.371954 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:51.371933 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerStarted","Data":"f90afa31e2f172b0c6cb622c851886ca677f4fce8dce4087287adde09d34e82e"} Jul 28 17:19:51.375811 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:51.375787 2570 generic.go:356] "Generic (PLEG): container finished" podID="774ef898-20b6-4061-a731-4e9cc83b5200" containerID="1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917" exitCode=0 Jul 28 17:19:51.375811 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:51.375810 2570 generic.go:356] "Generic (PLEG): container finished" podID="774ef898-20b6-4061-a731-4e9cc83b5200" containerID="3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d" exitCode=0 Jul 28 17:19:51.375986 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:51.375864 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerDied","Data":"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917"} Jul 28 17:19:51.375986 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:51.375896 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerDied","Data":"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d"} Jul 28 17:19:56.400777 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:56.400745 2570 generic.go:356] "Generic (PLEG): container finished" podID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerID="e0c3c774b89a3e6a73af9eea9a6b90e970af400ce25e64ec31fab634a2fa1b0c" exitCode=0 Jul 28 17:19:56.401171 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:56.400818 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerDied","Data":"e0c3c774b89a3e6a73af9eea9a6b90e970af400ce25e64ec31fab634a2fa1b0c"} Jul 28 17:19:56.870731 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:56.870676 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 28 17:19:57.079420 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:57.079372 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 28 17:19:57.407861 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:57.407817 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerStarted","Data":"e0f54a6c091e69cf94f8a67715df02a04c2d4977fe0fdd35a97deea03e081f0d"} Jul 28 17:19:58.966942 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:58.966899 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59b69bb7d8-nht6n" podUID="182a260d-ab01-47b7-b963-e3f4b96ebed1" containerName="console" containerID="cri-o://99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4" gracePeriod=15 Jul 28 17:19:59.216841 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.216803 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:19:59.237196 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.237124 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-trusted-ca-bundle\") pod \"182a260d-ab01-47b7-b963-e3f4b96ebed1\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " Jul 28 17:19:59.237196 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.237166 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-serving-cert\") pod \"182a260d-ab01-47b7-b963-e3f4b96ebed1\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " Jul 28 17:19:59.237398 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.237216 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-config\") pod \"182a260d-ab01-47b7-b963-e3f4b96ebed1\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " Jul 28 17:19:59.237398 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.237273 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtbmg\" (UniqueName: \"kubernetes.io/projected/182a260d-ab01-47b7-b963-e3f4b96ebed1-kube-api-access-wtbmg\") pod \"182a260d-ab01-47b7-b963-e3f4b96ebed1\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " Jul 28 17:19:59.237398 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.237314 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-oauth-serving-cert\") pod \"182a260d-ab01-47b7-b963-e3f4b96ebed1\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " Jul 28 17:19:59.237398 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.237390 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-service-ca\") pod \"182a260d-ab01-47b7-b963-e3f4b96ebed1\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " Jul 28 17:19:59.238238 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.237425 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-oauth-config\") pod \"182a260d-ab01-47b7-b963-e3f4b96ebed1\" (UID: \"182a260d-ab01-47b7-b963-e3f4b96ebed1\") " Jul 28 17:19:59.238355 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.238303 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-service-ca" (OuterVolumeSpecName: "service-ca") pod "182a260d-ab01-47b7-b963-e3f4b96ebed1" (UID: "182a260d-ab01-47b7-b963-e3f4b96ebed1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:59.238605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.238569 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "182a260d-ab01-47b7-b963-e3f4b96ebed1" (UID: "182a260d-ab01-47b7-b963-e3f4b96ebed1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:59.238727 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.238692 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "182a260d-ab01-47b7-b963-e3f4b96ebed1" (UID: "182a260d-ab01-47b7-b963-e3f4b96ebed1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:59.238873 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.238811 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-config" (OuterVolumeSpecName: "console-config") pod "182a260d-ab01-47b7-b963-e3f4b96ebed1" (UID: "182a260d-ab01-47b7-b963-e3f4b96ebed1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:59.242204 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.242171 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/182a260d-ab01-47b7-b963-e3f4b96ebed1-kube-api-access-wtbmg" (OuterVolumeSpecName: "kube-api-access-wtbmg") pod "182a260d-ab01-47b7-b963-e3f4b96ebed1" (UID: "182a260d-ab01-47b7-b963-e3f4b96ebed1"). InnerVolumeSpecName "kube-api-access-wtbmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:19:59.242389 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.242365 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "182a260d-ab01-47b7-b963-e3f4b96ebed1" (UID: "182a260d-ab01-47b7-b963-e3f4b96ebed1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:59.242487 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.242464 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "182a260d-ab01-47b7-b963-e3f4b96ebed1" (UID: "182a260d-ab01-47b7-b963-e3f4b96ebed1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:59.338125 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.338092 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-service-ca\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:19:59.338125 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.338123 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-oauth-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:19:59.338323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.338137 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-trusted-ca-bundle\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:19:59.338323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.338151 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:19:59.338323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.338162 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-console-config\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:19:59.338323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.338176 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtbmg\" (UniqueName: \"kubernetes.io/projected/182a260d-ab01-47b7-b963-e3f4b96ebed1-kube-api-access-wtbmg\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:19:59.338323 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.338186 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/182a260d-ab01-47b7-b963-e3f4b96ebed1-oauth-serving-cert\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:19:59.419479 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.419444 2570 generic.go:356] "Generic (PLEG): container finished" podID="182a260d-ab01-47b7-b963-e3f4b96ebed1" containerID="99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4" exitCode=0 Jul 28 17:19:59.419639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.419491 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59b69bb7d8-nht6n" Jul 28 17:19:59.419639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.419545 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b69bb7d8-nht6n" event={"ID":"182a260d-ab01-47b7-b963-e3f4b96ebed1","Type":"ContainerDied","Data":"99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4"} Jul 28 17:19:59.419639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.419608 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59b69bb7d8-nht6n" event={"ID":"182a260d-ab01-47b7-b963-e3f4b96ebed1","Type":"ContainerDied","Data":"1482eee1843f7b957897f28843c255ecf6e35ccd13a27c2bb7f658517a852b63"} Jul 28 17:19:59.419639 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.419637 2570 scope.go:117] "RemoveContainer" containerID="99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4" Jul 28 17:19:59.429134 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.429114 2570 scope.go:117] "RemoveContainer" containerID="99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4" Jul 28 17:19:59.429421 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:19:59.429402 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4\": container with ID starting with 99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4 not found: ID does not exist" containerID="99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4" Jul 28 17:19:59.429467 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.429431 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4"} err="failed to get container status \"99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4\": rpc error: code = NotFound desc = could not find container \"99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4\": container with ID starting with 99fe8e27182e2ba51f5e1827b0765786f2392cffc30917e39247ee9bfb6b17a4 not found: ID does not exist" Jul 28 17:19:59.442874 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.442843 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59b69bb7d8-nht6n"] Jul 28 17:19:59.446070 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.446045 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59b69bb7d8-nht6n"] Jul 28 17:19:59.724334 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:19:59.724293 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="182a260d-ab01-47b7-b963-e3f4b96ebed1" path="/var/lib/kubelet/pods/182a260d-ab01-47b7-b963-e3f4b96ebed1/volumes" Jul 28 17:20:02.436175 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:02.436089 2570 generic.go:356] "Generic (PLEG): container finished" podID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerID="e0f54a6c091e69cf94f8a67715df02a04c2d4977fe0fdd35a97deea03e081f0d" exitCode=0 Jul 28 17:20:02.436175 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:02.436160 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerDied","Data":"e0f54a6c091e69cf94f8a67715df02a04c2d4977fe0fdd35a97deea03e081f0d"} Jul 28 17:20:03.447600 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:03.447560 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerStarted","Data":"8002db6ad9cc512ef7e41c1f847cdb278577af72222057b18b4e08f99f36475f"} Jul 28 17:20:03.448107 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:03.447623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerStarted","Data":"8c9250d1f65cd503dc4dcec7a6aa2ba7eaca03eecb305bb8373dfa2cc272a88f"} Jul 28 17:20:03.448107 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:03.447645 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerStarted","Data":"b7a4dcac0be3484dbf61e07e34750b57bdb51e4ced58fcf9c1cc4a4834c9c5a7"} Jul 28 17:20:03.448107 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:03.447662 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerStarted","Data":"fb1a74b4c93d6a4fc90e134146a0726b97631e2809679718453d116d3cb77fc7"} Jul 28 17:20:03.481846 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:03.481764 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podStartSLOduration=13.48174171 podStartE2EDuration="13.48174171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:19:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:20:03.479252438 +0000 UTC m=+686.238113353" watchObservedRunningTime="2026-07-28 17:20:03.48174171 +0000 UTC m=+686.240602626" Jul 28 17:20:05.941962 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.941904 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:05.942523 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.942492 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:05.942523 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.942527 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:05.942709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.942544 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:05.944408 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.944367 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Jul 28 17:20:05.944539 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.944446 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 28 17:20:05.944539 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.944462 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.30:6571: connect: connection refused" Jul 28 17:20:05.944539 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:05.944367 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.30:8816: connect: connection refused" Jul 28 17:20:06.870667 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:06.870618 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 28 17:20:07.079837 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:07.079788 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 28 17:20:08.942465 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.942397 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Jul 28 17:20:08.943265 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.943234 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:08.943390 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.943354 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:08.943784 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.943757 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:08.943928 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.943801 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:08.943996 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.943963 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:08.944316 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.944296 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:08.948692 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:08.948673 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:09.493113 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:09.493084 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:09.496289 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:09.496263 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:10.942731 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:10.942693 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:11.942686 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:11.942652 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:12.508011 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:12.507976 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:20:16.870972 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:16.870924 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.29:8443: connect: connection refused" Jul 28 17:20:16.871509 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:16.871072 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:20:17.080067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:17.080020 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" probeResult="failure" output="Get \"https://10.132.0.29:6567/health\": dial tcp 10.132.0.29:6567: connect: connection refused" Jul 28 17:20:17.080258 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:17.080180 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:20:20.412131 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.412104 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:20:20.531187 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.531093 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/774ef898-20b6-4061-a731-4e9cc83b5200-feast-data\") pod \"774ef898-20b6-4061-a731-4e9cc83b5200\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " Jul 28 17:20:20.531187 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.531136 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6282\" (UniqueName: \"kubernetes.io/projected/774ef898-20b6-4061-a731-4e9cc83b5200-kube-api-access-f6282\") pod \"774ef898-20b6-4061-a731-4e9cc83b5200\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " Jul 28 17:20:20.531187 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.531163 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-ui-tls\") pod \"774ef898-20b6-4061-a731-4e9cc83b5200\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " Jul 28 17:20:20.531187 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.531188 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-online-tls\") pod \"774ef898-20b6-4061-a731-4e9cc83b5200\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " Jul 28 17:20:20.531480 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.531247 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-offline-tls\") pod \"774ef898-20b6-4061-a731-4e9cc83b5200\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " Jul 28 17:20:20.531480 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.531265 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-registry-tls\") pod \"774ef898-20b6-4061-a731-4e9cc83b5200\" (UID: \"774ef898-20b6-4061-a731-4e9cc83b5200\") " Jul 28 17:20:20.531777 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.531745 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/774ef898-20b6-4061-a731-4e9cc83b5200-feast-data" (OuterVolumeSpecName: "feast-data") pod "774ef898-20b6-4061-a731-4e9cc83b5200" (UID: "774ef898-20b6-4061-a731-4e9cc83b5200"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:20:20.533545 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.533514 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "774ef898-20b6-4061-a731-4e9cc83b5200" (UID: "774ef898-20b6-4061-a731-4e9cc83b5200"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:20.533658 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.533578 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "774ef898-20b6-4061-a731-4e9cc83b5200" (UID: "774ef898-20b6-4061-a731-4e9cc83b5200"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:20.533658 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.533630 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/774ef898-20b6-4061-a731-4e9cc83b5200-kube-api-access-f6282" (OuterVolumeSpecName: "kube-api-access-f6282") pod "774ef898-20b6-4061-a731-4e9cc83b5200" (UID: "774ef898-20b6-4061-a731-4e9cc83b5200"). InnerVolumeSpecName "kube-api-access-f6282". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:20:20.533775 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.533700 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-online-tls" (OuterVolumeSpecName: "online-tls") pod "774ef898-20b6-4061-a731-4e9cc83b5200" (UID: "774ef898-20b6-4061-a731-4e9cc83b5200"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:20.533775 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.533722 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "774ef898-20b6-4061-a731-4e9cc83b5200" (UID: "774ef898-20b6-4061-a731-4e9cc83b5200"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:20.549451 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.549421 2570 generic.go:356] "Generic (PLEG): container finished" podID="774ef898-20b6-4061-a731-4e9cc83b5200" containerID="bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23" exitCode=137 Jul 28 17:20:20.549451 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.549448 2570 generic.go:356] "Generic (PLEG): container finished" podID="774ef898-20b6-4061-a731-4e9cc83b5200" containerID="8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d" exitCode=137 Jul 28 17:20:20.549627 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.549491 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" Jul 28 17:20:20.549627 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.549499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerDied","Data":"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23"} Jul 28 17:20:20.549627 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.549549 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerDied","Data":"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d"} Jul 28 17:20:20.549627 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.549567 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q" event={"ID":"774ef898-20b6-4061-a731-4e9cc83b5200","Type":"ContainerDied","Data":"726aa7b4d01512d587fea818cfef5e99a18d07acdfbb0a8e06a68aa1d9057e6a"} Jul 28 17:20:20.549627 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.549584 2570 scope.go:117] "RemoveContainer" containerID="1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917" Jul 28 17:20:20.559371 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.559350 2570 scope.go:117] "RemoveContainer" containerID="bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23" Jul 28 17:20:20.568098 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.568076 2570 scope.go:117] "RemoveContainer" containerID="3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d" Jul 28 17:20:20.575436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.575409 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q"] Jul 28 17:20:20.576428 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.576414 2570 scope.go:117] "RemoveContainer" containerID="8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d" Jul 28 17:20:20.580944 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.580922 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-6tj4q"] Jul 28 17:20:20.585533 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.585512 2570 scope.go:117] "RemoveContainer" containerID="4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e" Jul 28 17:20:20.593623 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.593604 2570 scope.go:117] "RemoveContainer" containerID="da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f" Jul 28 17:20:20.601231 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.601207 2570 scope.go:117] "RemoveContainer" containerID="1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917" Jul 28 17:20:20.601501 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:20:20.601480 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917\": container with ID starting with 1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917 not found: ID does not exist" containerID="1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917" Jul 28 17:20:20.601552 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.601513 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917"} err="failed to get container status \"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917\": rpc error: code = NotFound desc = could not find container \"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917\": container with ID starting with 1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917 not found: ID does not exist" Jul 28 17:20:20.601552 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.601533 2570 scope.go:117] "RemoveContainer" containerID="bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23" Jul 28 17:20:20.601813 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:20:20.601796 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23\": container with ID starting with bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23 not found: ID does not exist" containerID="bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23" Jul 28 17:20:20.601875 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.601819 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23"} err="failed to get container status \"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23\": rpc error: code = NotFound desc = could not find container \"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23\": container with ID starting with bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23 not found: ID does not exist" Jul 28 17:20:20.601875 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.601857 2570 scope.go:117] "RemoveContainer" containerID="3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d" Jul 28 17:20:20.602097 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:20:20.602080 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d\": container with ID starting with 3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d not found: ID does not exist" containerID="3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d" Jul 28 17:20:20.602145 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602099 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d"} err="failed to get container status \"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d\": rpc error: code = NotFound desc = could not find container \"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d\": container with ID starting with 3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d not found: ID does not exist" Jul 28 17:20:20.602145 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602112 2570 scope.go:117] "RemoveContainer" containerID="8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d" Jul 28 17:20:20.602326 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:20:20.602310 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d\": container with ID starting with 8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d not found: ID does not exist" containerID="8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d" Jul 28 17:20:20.602376 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602329 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d"} err="failed to get container status \"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d\": rpc error: code = NotFound desc = could not find container \"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d\": container with ID starting with 8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d not found: ID does not exist" Jul 28 17:20:20.602376 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602344 2570 scope.go:117] "RemoveContainer" containerID="4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e" Jul 28 17:20:20.602528 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:20:20.602512 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e\": container with ID starting with 4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e not found: ID does not exist" containerID="4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e" Jul 28 17:20:20.602569 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602532 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e"} err="failed to get container status \"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e\": rpc error: code = NotFound desc = could not find container \"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e\": container with ID starting with 4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e not found: ID does not exist" Jul 28 17:20:20.602569 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602545 2570 scope.go:117] "RemoveContainer" containerID="da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f" Jul 28 17:20:20.602760 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:20:20.602746 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f\": container with ID starting with da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f not found: ID does not exist" containerID="da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f" Jul 28 17:20:20.602799 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602762 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f"} err="failed to get container status \"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f\": rpc error: code = NotFound desc = could not find container \"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f\": container with ID starting with da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f not found: ID does not exist" Jul 28 17:20:20.602799 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.602774 2570 scope.go:117] "RemoveContainer" containerID="1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917" Jul 28 17:20:20.603252 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603231 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917"} err="failed to get container status \"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917\": rpc error: code = NotFound desc = could not find container \"1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917\": container with ID starting with 1aecf6fced8d2f0cb69f30abda507a41e074472663ae72f7eab2b86019032917 not found: ID does not exist" Jul 28 17:20:20.603302 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603253 2570 scope.go:117] "RemoveContainer" containerID="bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23" Jul 28 17:20:20.603440 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603423 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23"} err="failed to get container status \"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23\": rpc error: code = NotFound desc = could not find container \"bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23\": container with ID starting with bf384902bf5557696b22bbb390a237629e2da9e8782ac83068d3d8e19e77fb23 not found: ID does not exist" Jul 28 17:20:20.603480 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603448 2570 scope.go:117] "RemoveContainer" containerID="3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d" Jul 28 17:20:20.603659 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603643 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d"} err="failed to get container status \"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d\": rpc error: code = NotFound desc = could not find container \"3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d\": container with ID starting with 3ba73498cad38ad1576c1902cee4650210eb7082aaac430154d527cfbe6e2d7d not found: ID does not exist" Jul 28 17:20:20.603715 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603659 2570 scope.go:117] "RemoveContainer" containerID="8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d" Jul 28 17:20:20.603862 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603844 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d"} err="failed to get container status \"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d\": rpc error: code = NotFound desc = could not find container \"8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d\": container with ID starting with 8a9461a186a8f4355cb98c48f18addf8088b764bc9d502e583424e8e8a85d49d not found: ID does not exist" Jul 28 17:20:20.603911 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.603863 2570 scope.go:117] "RemoveContainer" containerID="4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e" Jul 28 17:20:20.604053 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.604037 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e"} err="failed to get container status \"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e\": rpc error: code = NotFound desc = could not find container \"4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e\": container with ID starting with 4a5b70ac78f356c594d1a811887e4fefd8c60e874b4f7a2b7dc9b3036a102d3e not found: ID does not exist" Jul 28 17:20:20.604094 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.604052 2570 scope.go:117] "RemoveContainer" containerID="da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f" Jul 28 17:20:20.604235 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.604219 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f"} err="failed to get container status \"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f\": rpc error: code = NotFound desc = could not find container \"da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f\": container with ID starting with da290b1915f75396b608737265a20fb8082b86fe9744ff3fd7bffc4efbfdbc6f not found: ID does not exist" Jul 28 17:20:20.632706 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.632668 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-offline-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:20:20.632706 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.632701 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-registry-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:20:20.632706 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.632712 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/774ef898-20b6-4061-a731-4e9cc83b5200-feast-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:20:20.632962 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.632723 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6282\" (UniqueName: \"kubernetes.io/projected/774ef898-20b6-4061-a731-4e9cc83b5200-kube-api-access-f6282\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:20:20.632962 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.632735 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-ui-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:20:20.632962 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:20.632746 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/774ef898-20b6-4061-a731-4e9cc83b5200-online-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:20:21.729180 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:20:21.724756 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" path="/var/lib/kubelet/pods/774ef898-20b6-4061-a731-4e9cc83b5200/volumes" Jul 28 17:21:15.731914 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.731884 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9"] Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732249 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="feast-init" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732265 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="feast-init" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732276 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="registry" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732283 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="registry" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732307 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732316 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732330 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="feast-apply" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732337 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="feast-apply" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732348 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732357 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732367 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="offline" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732375 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="offline" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732386 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="182a260d-ab01-47b7-b963-e3f4b96ebed1" containerName="console" Jul 28 17:21:15.732436 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732391 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="182a260d-ab01-47b7-b963-e3f4b96ebed1" containerName="console" Jul 28 17:21:15.732918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732459 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="online" Jul 28 17:21:15.732918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732469 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="offline" Jul 28 17:21:15.732918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732474 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="182a260d-ab01-47b7-b963-e3f4b96ebed1" containerName="console" Jul 28 17:21:15.732918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732480 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="registry" Jul 28 17:21:15.732918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.732489 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="774ef898-20b6-4061-a731-4e9cc83b5200" containerName="ui" Jul 28 17:21:15.736124 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.736108 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.739687 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.739662 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 28 17:21:15.739800 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.739772 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Jul 28 17:21:15.740002 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.739980 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 28 17:21:15.740002 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.739998 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 28 17:21:15.740322 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.740303 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 28 17:21:15.740394 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.740323 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-k2pt8\"" Jul 28 17:21:15.740394 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.740367 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 28 17:21:15.740684 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.740668 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:21:15.758301 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.758271 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9"] Jul 28 17:21:15.791513 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.791483 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mp4w\" (UniqueName: \"kubernetes.io/projected/7245870e-fe49-4ae2-85b8-559faed72fdf-kube-api-access-5mp4w\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.791668 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.791532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.791668 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.791552 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.791758 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.791715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7245870e-fe49-4ae2-85b8-559faed72fdf-feast-data\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.791849 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.791810 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7245870e-fe49-4ae2-85b8-559faed72fdf-registry-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.791949 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.791932 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-offline-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.892852 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.892789 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mp4w\" (UniqueName: \"kubernetes.io/projected/7245870e-fe49-4ae2-85b8-559faed72fdf-kube-api-access-5mp4w\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.892852 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.892858 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.893073 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.892885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.893073 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:21:15.892972 2570 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 28 17:21:15.893073 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:21:15.893022 2570 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found Jul 28 17:21:15.893073 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.893042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7245870e-fe49-4ae2-85b8-559faed72fdf-feast-data\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.893073 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:21:15.893052 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls podName:7245870e-fe49-4ae2-85b8-559faed72fdf nodeName:}" failed. No retries permitted until 2026-07-28 17:21:16.393029599 +0000 UTC m=+759.151890492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls") pod "feast-simple-feast-remote-setup-578b464648-8fxf9" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf") : secret "feast-simple-feast-remote-setup-ui-tls" not found Jul 28 17:21:15.893336 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:21:15.893084 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls podName:7245870e-fe49-4ae2-85b8-559faed72fdf nodeName:}" failed. No retries permitted until 2026-07-28 17:21:16.393072362 +0000 UTC m=+759.151933254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls") pod "feast-simple-feast-remote-setup-578b464648-8fxf9" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf") : secret "feast-simple-feast-remote-setup-online-tls" not found Jul 28 17:21:15.893336 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.893157 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7245870e-fe49-4ae2-85b8-559faed72fdf-registry-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.893336 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.893215 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-offline-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.893493 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.893449 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7245870e-fe49-4ae2-85b8-559faed72fdf-feast-data\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.893765 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.893743 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7245870e-fe49-4ae2-85b8-559faed72fdf-registry-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.895553 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.895526 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-offline-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:15.903037 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:15.903011 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mp4w\" (UniqueName: \"kubernetes.io/projected/7245870e-fe49-4ae2-85b8-559faed72fdf-kube-api-access-5mp4w\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:16.397727 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:16.397683 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:16.397727 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:16.397731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:16.400117 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:16.400083 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:16.400255 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:16.400174 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls\") pod \"feast-simple-feast-remote-setup-578b464648-8fxf9\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:16.646537 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:16.646479 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:16.783667 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:16.783641 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9"] Jul 28 17:21:16.785465 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:21:16.785423 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7245870e_fe49_4ae2_85b8_559faed72fdf.slice/crio-f4331e1b01673907fa1949a055a30986bcc058fec68fa3285e8b3f860c092d3f WatchSource:0}: Error finding container f4331e1b01673907fa1949a055a30986bcc058fec68fa3285e8b3f860c092d3f: Status 404 returned error can't find the container with id f4331e1b01673907fa1949a055a30986bcc058fec68fa3285e8b3f860c092d3f Jul 28 17:21:17.795985 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:17.795946 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerStarted","Data":"5545b35d6b125ca8dfc3405a3e61611c33c843728617cd7318c8d6b69a50c355"} Jul 28 17:21:17.796432 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:17.795998 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerStarted","Data":"f4331e1b01673907fa1949a055a30986bcc058fec68fa3285e8b3f860c092d3f"} Jul 28 17:21:21.814792 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:21.814759 2570 generic.go:356] "Generic (PLEG): container finished" podID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerID="5545b35d6b125ca8dfc3405a3e61611c33c843728617cd7318c8d6b69a50c355" exitCode=0 Jul 28 17:21:21.815307 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:21.814857 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerDied","Data":"5545b35d6b125ca8dfc3405a3e61611c33c843728617cd7318c8d6b69a50c355"} Jul 28 17:21:22.822325 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:22.822291 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerStarted","Data":"29aaf601ba6a8d38d803b185a9cd8d9548f71645bb308479c4ae1e8815634ac5"} Jul 28 17:21:28.850185 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:28.850144 2570 generic.go:356] "Generic (PLEG): container finished" podID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerID="29aaf601ba6a8d38d803b185a9cd8d9548f71645bb308479c4ae1e8815634ac5" exitCode=0 Jul 28 17:21:28.850583 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:28.850216 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerDied","Data":"29aaf601ba6a8d38d803b185a9cd8d9548f71645bb308479c4ae1e8815634ac5"} Jul 28 17:21:29.858788 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:29.858745 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerStarted","Data":"0a4f63d3c7f36e8bc655c7e7d5c19a24e10540385e5dc738eff75c1c5e006fd2"} Jul 28 17:21:29.859165 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:29.858813 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerStarted","Data":"49e004e57abbaee8c88b31c4e547600b35da748982e3fe6e15e2c8bc692547c7"} Jul 28 17:21:29.859165 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:29.858858 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerStarted","Data":"1f41fe4cbb941314939f77307fb79074336ae6577f7e5f3d6c71334c72ed8264"} Jul 28 17:21:31.646679 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:31.646642 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:31.647264 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:31.647233 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:31.647264 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:31.647263 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:31.648703 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:31.648669 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.31:8816: connect: connection refused" Jul 28 17:21:31.648822 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:31.648703 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 28 17:21:31.648822 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:31.648706 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Jul 28 17:21:34.647380 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.647334 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Jul 28 17:21:34.648312 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.648288 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:34.648783 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.648757 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:34.651019 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.651000 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:34.673162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.673102 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podStartSLOduration=19.6730809 podStartE2EDuration="19.6730809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:21:29.896748402 +0000 UTC m=+772.655609317" watchObservedRunningTime="2026-07-28 17:21:34.6730809 +0000 UTC m=+777.431941815" Jul 28 17:21:34.848430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.848401 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:34.887117 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.887088 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:34.891849 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:34.891798 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:36.646753 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:36.646711 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:37.647475 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:37.647433 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:21:37.901651 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:21:37.901564 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:22:39.100988 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.100955 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9"] Jul 28 17:22:39.101502 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.101189 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" containerID="cri-o://1f41fe4cbb941314939f77307fb79074336ae6577f7e5f3d6c71334c72ed8264" gracePeriod=30 Jul 28 17:22:39.101502 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.101274 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="offline" containerID="cri-o://49e004e57abbaee8c88b31c4e547600b35da748982e3fe6e15e2c8bc692547c7" gracePeriod=30 Jul 28 17:22:39.101502 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.101273 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" containerID="cri-o://0a4f63d3c7f36e8bc655c7e7d5c19a24e10540385e5dc738eff75c1c5e006fd2" gracePeriod=30 Jul 28 17:22:39.203546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.203515 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct"] Jul 28 17:22:39.203815 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.203760 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="registry" containerID="cri-o://fb1a74b4c93d6a4fc90e134146a0726b97631e2809679718453d116d3cb77fc7" gracePeriod=30 Jul 28 17:22:39.203998 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.203815 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" containerID="cri-o://8002db6ad9cc512ef7e41c1f847cdb278577af72222057b18b4e08f99f36475f" gracePeriod=30 Jul 28 17:22:39.203998 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.203796 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="offline" containerID="cri-o://8c9250d1f65cd503dc4dcec7a6aa2ba7eaca03eecb305bb8373dfa2cc272a88f" gracePeriod=30 Jul 28 17:22:39.203998 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.203818 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" containerID="cri-o://b7a4dcac0be3484dbf61e07e34750b57bdb51e4ced58fcf9c1cc4a4834c9c5a7" gracePeriod=30 Jul 28 17:22:39.493463 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:39.493358 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 28 17:22:40.180311 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.180273 2570 generic.go:356] "Generic (PLEG): container finished" podID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerID="8002db6ad9cc512ef7e41c1f847cdb278577af72222057b18b4e08f99f36475f" exitCode=0 Jul 28 17:22:40.180311 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.180304 2570 generic.go:356] "Generic (PLEG): container finished" podID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerID="b7a4dcac0be3484dbf61e07e34750b57bdb51e4ced58fcf9c1cc4a4834c9c5a7" exitCode=0 Jul 28 17:22:40.180767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.180337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerDied","Data":"8002db6ad9cc512ef7e41c1f847cdb278577af72222057b18b4e08f99f36475f"} Jul 28 17:22:40.180767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.180380 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerDied","Data":"b7a4dcac0be3484dbf61e07e34750b57bdb51e4ced58fcf9c1cc4a4834c9c5a7"} Jul 28 17:22:40.183151 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.183124 2570 generic.go:356] "Generic (PLEG): container finished" podID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerID="0a4f63d3c7f36e8bc655c7e7d5c19a24e10540385e5dc738eff75c1c5e006fd2" exitCode=0 Jul 28 17:22:40.183151 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.183145 2570 generic.go:356] "Generic (PLEG): container finished" podID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerID="1f41fe4cbb941314939f77307fb79074336ae6577f7e5f3d6c71334c72ed8264" exitCode=0 Jul 28 17:22:40.183301 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.183164 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerDied","Data":"0a4f63d3c7f36e8bc655c7e7d5c19a24e10540385e5dc738eff75c1c5e006fd2"} Jul 28 17:22:40.183301 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:40.183193 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerDied","Data":"1f41fe4cbb941314939f77307fb79074336ae6577f7e5f3d6c71334c72ed8264"} Jul 28 17:22:42.508536 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:42.508490 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Jul 28 17:22:44.887459 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:44.887421 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 28 17:22:47.901220 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:47.901176 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Jul 28 17:22:49.493533 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:49.493482 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 28 17:22:52.508138 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:52.508100 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Jul 28 17:22:54.887546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:54.887501 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 28 17:22:57.902011 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:57.901970 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Jul 28 17:22:59.494132 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:59.494087 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 28 17:22:59.494598 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:22:59.494224 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:23:02.507738 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:02.507694 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.30:8443: connect: connection refused" Jul 28 17:23:02.508251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:02.507856 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:23:04.888141 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:04.888096 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" probeResult="failure" output="Get \"https://10.132.0.31:6567/health\": dial tcp 10.132.0.31:6567: connect: connection refused" Jul 28 17:23:04.888538 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:04.888230 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:23:07.901528 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:07.901483 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" probeResult="failure" output="dial tcp 10.132.0.31:8443: connect: connection refused" Jul 28 17:23:07.902016 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:07.901618 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:23:09.315767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.315725 2570 generic.go:356] "Generic (PLEG): container finished" podID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerID="8c9250d1f65cd503dc4dcec7a6aa2ba7eaca03eecb305bb8373dfa2cc272a88f" exitCode=137 Jul 28 17:23:09.315767 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.315760 2570 generic.go:356] "Generic (PLEG): container finished" podID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerID="fb1a74b4c93d6a4fc90e134146a0726b97631e2809679718453d116d3cb77fc7" exitCode=137 Jul 28 17:23:09.316221 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.315789 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerDied","Data":"8c9250d1f65cd503dc4dcec7a6aa2ba7eaca03eecb305bb8373dfa2cc272a88f"} Jul 28 17:23:09.316221 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.315842 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerDied","Data":"fb1a74b4c93d6a4fc90e134146a0726b97631e2809679718453d116d3cb77fc7"} Jul 28 17:23:09.318503 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.318479 2570 generic.go:356] "Generic (PLEG): container finished" podID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerID="49e004e57abbaee8c88b31c4e547600b35da748982e3fe6e15e2c8bc692547c7" exitCode=137 Jul 28 17:23:09.318611 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.318550 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerDied","Data":"49e004e57abbaee8c88b31c4e547600b35da748982e3fe6e15e2c8bc692547c7"} Jul 28 17:23:09.318611 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.318587 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" event={"ID":"7245870e-fe49-4ae2-85b8-559faed72fdf","Type":"ContainerDied","Data":"f4331e1b01673907fa1949a055a30986bcc058fec68fa3285e8b3f860c092d3f"} Jul 28 17:23:09.318611 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.318601 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4331e1b01673907fa1949a055a30986bcc058fec68fa3285e8b3f860c092d3f" Jul 28 17:23:09.324016 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.323995 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:23:09.485248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485153 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-offline-tls\") pod \"7245870e-fe49-4ae2-85b8-559faed72fdf\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " Jul 28 17:23:09.485248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485215 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7245870e-fe49-4ae2-85b8-559faed72fdf-feast-data\") pod \"7245870e-fe49-4ae2-85b8-559faed72fdf\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " Jul 28 17:23:09.485248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485237 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mp4w\" (UniqueName: \"kubernetes.io/projected/7245870e-fe49-4ae2-85b8-559faed72fdf-kube-api-access-5mp4w\") pod \"7245870e-fe49-4ae2-85b8-559faed72fdf\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " Jul 28 17:23:09.485531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485342 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls\") pod \"7245870e-fe49-4ae2-85b8-559faed72fdf\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " Jul 28 17:23:09.485531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485388 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7245870e-fe49-4ae2-85b8-559faed72fdf-registry-tls\") pod \"7245870e-fe49-4ae2-85b8-559faed72fdf\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " Jul 28 17:23:09.485531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485418 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls\") pod \"7245870e-fe49-4ae2-85b8-559faed72fdf\" (UID: \"7245870e-fe49-4ae2-85b8-559faed72fdf\") " Jul 28 17:23:09.485858 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485814 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7245870e-fe49-4ae2-85b8-559faed72fdf-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7245870e-fe49-4ae2-85b8-559faed72fdf" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:23:09.486004 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.485975 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7245870e-fe49-4ae2-85b8-559faed72fdf-feast-data" (OuterVolumeSpecName: "feast-data") pod "7245870e-fe49-4ae2-85b8-559faed72fdf" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:23:09.487571 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.487539 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "7245870e-fe49-4ae2-85b8-559faed72fdf" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:09.487672 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.487611 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7245870e-fe49-4ae2-85b8-559faed72fdf-kube-api-access-5mp4w" (OuterVolumeSpecName: "kube-api-access-5mp4w") pod "7245870e-fe49-4ae2-85b8-559faed72fdf" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf"). InnerVolumeSpecName "kube-api-access-5mp4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:23:09.487672 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.487634 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls" (OuterVolumeSpecName: "online-tls") pod "7245870e-fe49-4ae2-85b8-559faed72fdf" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:09.487672 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.487645 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7245870e-fe49-4ae2-85b8-559faed72fdf" (UID: "7245870e-fe49-4ae2-85b8-559faed72fdf"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:09.494147 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.494105 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" probeResult="failure" output="Get \"https://10.132.0.30:6567/health\": dial tcp 10.132.0.30:6567: connect: connection refused" Jul 28 17:23:09.586769 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.586722 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-offline-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:09.586769 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.586763 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7245870e-fe49-4ae2-85b8-559faed72fdf-feast-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:09.586769 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.586775 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mp4w\" (UniqueName: \"kubernetes.io/projected/7245870e-fe49-4ae2-85b8-559faed72fdf-kube-api-access-5mp4w\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:09.586769 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.586785 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-ui-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:09.587051 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.586793 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/7245870e-fe49-4ae2-85b8-559faed72fdf-registry-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:09.587051 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.586802 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7245870e-fe49-4ae2-85b8-559faed72fdf-online-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:09.854029 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.854000 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:23:09.990362 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.990332 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-online-tls\") pod \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " Jul 28 17:23:09.990546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.990375 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls\") pod \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " Jul 28 17:23:09.990546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.990418 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-registry-tls\") pod \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " Jul 28 17:23:09.990546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.990435 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-feast-data\") pod \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " Jul 28 17:23:09.990546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.990457 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5svb\" (UniqueName: \"kubernetes.io/projected/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-kube-api-access-v5svb\") pod \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " Jul 28 17:23:09.990546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.990481 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls\") pod \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\" (UID: \"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec\") " Jul 28 17:23:09.991080 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.991047 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-feast-data" (OuterVolumeSpecName: "feast-data") pod "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:23:09.992586 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.992567 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:09.993115 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.993083 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:09.993115 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.993097 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-online-tls" (OuterVolumeSpecName: "online-tls") pod "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:09.993233 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.993124 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:09.993233 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:09.993148 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-kube-api-access-v5svb" (OuterVolumeSpecName: "kube-api-access-v5svb") pod "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" (UID: "4e17ffd4-d4b4-4718-a48c-03d0f16c23ec"). InnerVolumeSpecName "kube-api-access-v5svb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:23:10.091067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.091044 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-offline-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.091067 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.091070 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-registry-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.091248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.091079 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-feast-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.091248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.091087 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5svb\" (UniqueName: \"kubernetes.io/projected/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-kube-api-access-v5svb\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.091248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.091096 2570 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-ui-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.091248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.091106 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec-online-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:23:10.326528 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.326500 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" Jul 28 17:23:10.326918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.326502 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9" Jul 28 17:23:10.326918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.326503 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct" event={"ID":"4e17ffd4-d4b4-4718-a48c-03d0f16c23ec","Type":"ContainerDied","Data":"f90afa31e2f172b0c6cb622c851886ca677f4fce8dce4087287adde09d34e82e"} Jul 28 17:23:10.326918 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.326640 2570 scope.go:117] "RemoveContainer" containerID="8002db6ad9cc512ef7e41c1f847cdb278577af72222057b18b4e08f99f36475f" Jul 28 17:23:10.335973 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.335949 2570 scope.go:117] "RemoveContainer" containerID="8c9250d1f65cd503dc4dcec7a6aa2ba7eaca03eecb305bb8373dfa2cc272a88f" Jul 28 17:23:10.343982 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.343969 2570 scope.go:117] "RemoveContainer" containerID="b7a4dcac0be3484dbf61e07e34750b57bdb51e4ced58fcf9c1cc4a4834c9c5a7" Jul 28 17:23:10.354104 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.354073 2570 scope.go:117] "RemoveContainer" containerID="fb1a74b4c93d6a4fc90e134146a0726b97631e2809679718453d116d3cb77fc7" Jul 28 17:23:10.355588 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.355566 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct"] Jul 28 17:23:10.363247 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.363228 2570 scope.go:117] "RemoveContainer" containerID="e0f54a6c091e69cf94f8a67715df02a04c2d4977fe0fdd35a97deea03e081f0d" Jul 28 17:23:10.363940 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.363917 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-56cb884f74-lg2ct"] Jul 28 17:23:10.371414 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.371398 2570 scope.go:117] "RemoveContainer" containerID="e0c3c774b89a3e6a73af9eea9a6b90e970af400ce25e64ec31fab634a2fa1b0c" Jul 28 17:23:10.384732 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.384709 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9"] Jul 28 17:23:10.392512 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:10.392489 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-578b464648-8fxf9"] Jul 28 17:23:11.723656 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:11.723613 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" path="/var/lib/kubelet/pods/4e17ffd4-d4b4-4718-a48c-03d0f16c23ec/volumes" Jul 28 17:23:11.724262 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:11.724248 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" path="/var/lib/kubelet/pods/7245870e-fe49-4ae2-85b8-559faed72fdf/volumes" Jul 28 17:23:22.695523 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.695489 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-95mnn"] Jul 28 17:23:22.696008 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.695991 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" Jul 28 17:23:22.696054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696011 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" Jul 28 17:23:22.696054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696028 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="offline" Jul 28 17:23:22.696054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696036 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="offline" Jul 28 17:23:22.696054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696046 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696055 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696068 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="feast-apply" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696077 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="feast-apply" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696090 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="feast-apply" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696098 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="feast-apply" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696112 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696122 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696134 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="registry" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696142 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="registry" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696151 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="offline" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696160 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="offline" Jul 28 17:23:22.696170 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696174 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="feast-init" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696183 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="feast-init" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696194 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696202 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696220 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="feast-init" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696229 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="feast-init" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696305 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="registry" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696319 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="offline" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696327 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="ui" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696338 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="online" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696346 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="ui" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696354 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e17ffd4-d4b4-4718-a48c-03d0f16c23ec" containerName="offline" Jul 28 17:23:22.696510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.696361 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7245870e-fe49-4ae2-85b8-559faed72fdf" containerName="online" Jul 28 17:23:22.700953 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.700929 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:22.703367 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.703346 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-5jhsm\"" Jul 28 17:23:22.703683 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.703666 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Jul 28 17:23:22.704131 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.704095 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 28 17:23:22.704131 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.704125 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 28 17:23:22.705043 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.705017 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-95mnn"] Jul 28 17:23:22.802228 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.802188 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qqck\" (UniqueName: \"kubernetes.io/projected/7330d427-24eb-401e-a40e-1fb9765e90f5-kube-api-access-4qqck\") pod \"postgres-864846d4f5-95mnn\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:22.802415 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.802240 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7330d427-24eb-401e-a40e-1fb9765e90f5-postgresdata\") pod \"postgres-864846d4f5-95mnn\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:22.903428 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.903390 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qqck\" (UniqueName: \"kubernetes.io/projected/7330d427-24eb-401e-a40e-1fb9765e90f5-kube-api-access-4qqck\") pod \"postgres-864846d4f5-95mnn\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:22.903573 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.903443 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7330d427-24eb-401e-a40e-1fb9765e90f5-postgresdata\") pod \"postgres-864846d4f5-95mnn\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:22.903807 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.903791 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7330d427-24eb-401e-a40e-1fb9765e90f5-postgresdata\") pod \"postgres-864846d4f5-95mnn\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:22.911736 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:22.911699 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qqck\" (UniqueName: \"kubernetes.io/projected/7330d427-24eb-401e-a40e-1fb9765e90f5-kube-api-access-4qqck\") pod \"postgres-864846d4f5-95mnn\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:23.019377 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:23.019291 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:23:23.158433 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:23.158408 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-95mnn"] Jul 28 17:23:23.161202 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:23:23.161175 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7330d427_24eb_401e_a40e_1fb9765e90f5.slice/crio-845146e41662ae6557906e38952dc7e8ecae293bb237d4f5a4dce6cc3bc79f02 WatchSource:0}: Error finding container 845146e41662ae6557906e38952dc7e8ecae293bb237d4f5a4dce6cc3bc79f02: Status 404 returned error can't find the container with id 845146e41662ae6557906e38952dc7e8ecae293bb237d4f5a4dce6cc3bc79f02 Jul 28 17:23:23.162998 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:23.162980 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:23:23.396095 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:23.396062 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-95mnn" event={"ID":"7330d427-24eb-401e-a40e-1fb9765e90f5","Type":"ContainerStarted","Data":"845146e41662ae6557906e38952dc7e8ecae293bb237d4f5a4dce6cc3bc79f02"} Jul 28 17:23:29.426535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:29.426493 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-95mnn" event={"ID":"7330d427-24eb-401e-a40e-1fb9765e90f5","Type":"ContainerStarted","Data":"6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278"} Jul 28 17:23:29.443712 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:29.443665 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-95mnn" podStartSLOduration=2.125994724 podStartE2EDuration="7.443648061s" totalImagesPullingTime="5.317653337s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:29.442468845 +0000 UTC m=+892.201329760" watchObservedRunningTime="2026-07-28 17:23:29.443648061 +0000 UTC m=+892.202508974" Jul 28 17:23:33.330404 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.330323 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-58d77f89-zhp9z"] Jul 28 17:23:33.334660 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.334640 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.337404 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.337384 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Jul 28 17:23:33.338536 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.338495 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Jul 28 17:23:33.338536 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.338503 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Jul 28 17:23:33.339011 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.338994 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-tdzbp\"" Jul 28 17:23:33.339109 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.339091 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 28 17:23:33.349568 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.349539 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-58d77f89-zhp9z"] Jul 28 17:23:33.391028 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.390999 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.391028 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.391031 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.391218 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.391087 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/586d1507-c8ff-4abc-88a7-639139252e78-feast-data\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.391218 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.391130 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsbch\" (UniqueName: \"kubernetes.io/projected/586d1507-c8ff-4abc-88a7-639139252e78-kube-api-access-qsbch\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.391218 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.391154 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.492035 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.491995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.492035 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.492035 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.492088 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/586d1507-c8ff-4abc-88a7-639139252e78-feast-data\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.492141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsbch\" (UniqueName: \"kubernetes.io/projected/586d1507-c8ff-4abc-88a7-639139252e78-kube-api-access-qsbch\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:23:33.492164 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.492177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:23:33.492229 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls podName:586d1507-c8ff-4abc-88a7-639139252e78 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:33.992207576 +0000 UTC m=+896.751068489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls") pod "feast-credit-scoring-58d77f89-zhp9z" (UID: "586d1507-c8ff-4abc-88a7-639139252e78") : secret "feast-credit-scoring-registry-tls" not found Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:23:33.492259 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-offline-tls: secret "feast-credit-scoring-offline-tls" not found Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:23:33.492302 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls podName:586d1507-c8ff-4abc-88a7-639139252e78 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:33.99229015 +0000 UTC m=+896.751151055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls") pod "feast-credit-scoring-58d77f89-zhp9z" (UID: "586d1507-c8ff-4abc-88a7-639139252e78") : secret "feast-credit-scoring-offline-tls" not found Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:23:33.492298 2570 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Jul 28 17:23:33.492461 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:23:33.492325 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls podName:586d1507-c8ff-4abc-88a7-639139252e78 nodeName:}" failed. No retries permitted until 2026-07-28 17:23:33.992320591 +0000 UTC m=+896.751181484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls") pod "feast-credit-scoring-58d77f89-zhp9z" (UID: "586d1507-c8ff-4abc-88a7-639139252e78") : secret "feast-credit-scoring-online-tls" not found Jul 28 17:23:33.492813 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.492501 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/586d1507-c8ff-4abc-88a7-639139252e78-feast-data\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.506788 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.506750 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsbch\" (UniqueName: \"kubernetes.io/projected/586d1507-c8ff-4abc-88a7-639139252e78-kube-api-access-qsbch\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.997201 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.997158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.997201 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.997202 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.997447 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.997297 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:33.999905 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.999874 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:34.000012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.999882 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:34.000012 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:33.999923 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls\") pod \"feast-credit-scoring-58d77f89-zhp9z\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:34.244554 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:34.244502 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:34.383896 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:34.383870 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-58d77f89-zhp9z"] Jul 28 17:23:34.384931 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:23:34.384906 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod586d1507_c8ff_4abc_88a7_639139252e78.slice/crio-d5973f9926ba7c77937ca322bb1dfaae79cb6d2cd2d56d012b98630b4c1680b6 WatchSource:0}: Error finding container d5973f9926ba7c77937ca322bb1dfaae79cb6d2cd2d56d012b98630b4c1680b6: Status 404 returned error can't find the container with id d5973f9926ba7c77937ca322bb1dfaae79cb6d2cd2d56d012b98630b4c1680b6 Jul 28 17:23:34.450260 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:34.450220 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerStarted","Data":"89fc0d1044a667ca8c94ac898c3e8baa77b514d4baf14c12f31e09a560bdd8df"} Jul 28 17:23:34.450425 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:34.450295 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerStarted","Data":"d5973f9926ba7c77937ca322bb1dfaae79cb6d2cd2d56d012b98630b4c1680b6"} Jul 28 17:23:35.457044 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:35.457004 2570 generic.go:356] "Generic (PLEG): container finished" podID="586d1507-c8ff-4abc-88a7-639139252e78" containerID="89fc0d1044a667ca8c94ac898c3e8baa77b514d4baf14c12f31e09a560bdd8df" exitCode=0 Jul 28 17:23:35.457479 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:35.457097 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerDied","Data":"89fc0d1044a667ca8c94ac898c3e8baa77b514d4baf14c12f31e09a560bdd8df"} Jul 28 17:23:36.465072 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:36.465035 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerStarted","Data":"2649612b5687ee1cb0feede97e904297a2a8d851503857518700623dca1c1093"} Jul 28 17:23:42.495766 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:42.495730 2570 generic.go:356] "Generic (PLEG): container finished" podID="586d1507-c8ff-4abc-88a7-639139252e78" containerID="2649612b5687ee1cb0feede97e904297a2a8d851503857518700623dca1c1093" exitCode=0 Jul 28 17:23:42.496208 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:42.495803 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerDied","Data":"2649612b5687ee1cb0feede97e904297a2a8d851503857518700623dca1c1093"} Jul 28 17:23:42.499876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:42.499851 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 28 17:23:43.504434 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:43.504396 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerStarted","Data":"e71f33b88c624ed8e3cc82fd66b1ac05a631ebb02150bc058940c446382d2621"} Jul 28 17:23:43.504954 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:43.504456 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerStarted","Data":"546b4d34704042997ef7a88b4d8c4d7b6d3a79ab99bb3ca551ff8b63e170c982"} Jul 28 17:23:43.504954 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:43.504474 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerStarted","Data":"bdde7e88c94ae5b03946ab52854f07b5303683df9d600b6404f8a3d679037ac7"} Jul 28 17:23:43.530011 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:43.529946 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podStartSLOduration=10.529923267000001 podStartE2EDuration="10.529923267s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:43.528711234 +0000 UTC m=+906.287572173" watchObservedRunningTime="2026-07-28 17:23:43.529923267 +0000 UTC m=+906.288784182" Jul 28 17:23:44.244788 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:44.244753 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:44.245014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:44.244820 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:44.245014 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:44.244849 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:46.244876 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:46.244810 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:46.245364 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:46.244914 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:46.245364 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:46.244944 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:46.246413 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:46.246373 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Jul 28 17:23:46.246413 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:46.246399 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="offline" probeResult="failure" output="dial tcp 10.132.0.33:8816: connect: connection refused" Jul 28 17:23:46.246586 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:46.246426 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 28 17:23:49.245805 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:49.245755 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Jul 28 17:23:49.246765 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:49.246743 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:49.247513 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:49.247493 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:49.253546 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:49.253526 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:49.543572 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:49.543482 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:52.245547 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:52.245506 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.33:6571: connect: connection refused" Jul 28 17:23:55.246081 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:55.246045 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:23:55.246842 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:23:55.246805 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:25:01.842356 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:01.842272 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-58d77f89-zhp9z"] Jul 28 17:25:01.842940 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:01.842508 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="registry" containerID="cri-o://bdde7e88c94ae5b03946ab52854f07b5303683df9d600b6404f8a3d679037ac7" gracePeriod=30 Jul 28 17:25:01.842940 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:01.842572 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" containerID="cri-o://546b4d34704042997ef7a88b4d8c4d7b6d3a79ab99bb3ca551ff8b63e170c982" gracePeriod=30 Jul 28 17:25:01.842940 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:01.842575 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="offline" containerID="cri-o://e71f33b88c624ed8e3cc82fd66b1ac05a631ebb02150bc058940c446382d2621" gracePeriod=30 Jul 28 17:25:01.854365 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:01.854333 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-95mnn"] Jul 28 17:25:01.854514 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:01.854495 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-95mnn" podUID="7330d427-24eb-401e-a40e-1fb9765e90f5" containerName="postgres" containerID="cri-o://6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278" gracePeriod=30 Jul 28 17:25:02.108581 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.108555 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:25:02.165368 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.165328 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qqck\" (UniqueName: \"kubernetes.io/projected/7330d427-24eb-401e-a40e-1fb9765e90f5-kube-api-access-4qqck\") pod \"7330d427-24eb-401e-a40e-1fb9765e90f5\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " Jul 28 17:25:02.165368 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.165365 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7330d427-24eb-401e-a40e-1fb9765e90f5-postgresdata\") pod \"7330d427-24eb-401e-a40e-1fb9765e90f5\" (UID: \"7330d427-24eb-401e-a40e-1fb9765e90f5\") " Jul 28 17:25:02.167459 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.167427 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7330d427-24eb-401e-a40e-1fb9765e90f5-kube-api-access-4qqck" (OuterVolumeSpecName: "kube-api-access-4qqck") pod "7330d427-24eb-401e-a40e-1fb9765e90f5" (UID: "7330d427-24eb-401e-a40e-1fb9765e90f5"). InnerVolumeSpecName "kube-api-access-4qqck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:25:02.195958 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.195916 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7330d427-24eb-401e-a40e-1fb9765e90f5-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "7330d427-24eb-401e-a40e-1fb9765e90f5" (UID: "7330d427-24eb-401e-a40e-1fb9765e90f5"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:25:02.266278 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.266232 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qqck\" (UniqueName: \"kubernetes.io/projected/7330d427-24eb-401e-a40e-1fb9765e90f5-kube-api-access-4qqck\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:25:02.266278 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.266269 2570 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/7330d427-24eb-401e-a40e-1fb9765e90f5-postgresdata\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:25:02.901256 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.901220 2570 generic.go:356] "Generic (PLEG): container finished" podID="586d1507-c8ff-4abc-88a7-639139252e78" containerID="546b4d34704042997ef7a88b4d8c4d7b6d3a79ab99bb3ca551ff8b63e170c982" exitCode=0 Jul 28 17:25:02.901676 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.901289 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerDied","Data":"546b4d34704042997ef7a88b4d8c4d7b6d3a79ab99bb3ca551ff8b63e170c982"} Jul 28 17:25:02.902484 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.902467 2570 generic.go:356] "Generic (PLEG): container finished" podID="7330d427-24eb-401e-a40e-1fb9765e90f5" containerID="6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278" exitCode=0 Jul 28 17:25:02.902551 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.902511 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-95mnn" Jul 28 17:25:02.902551 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.902522 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-95mnn" event={"ID":"7330d427-24eb-401e-a40e-1fb9765e90f5","Type":"ContainerDied","Data":"6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278"} Jul 28 17:25:02.902617 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.902561 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-95mnn" event={"ID":"7330d427-24eb-401e-a40e-1fb9765e90f5","Type":"ContainerDied","Data":"845146e41662ae6557906e38952dc7e8ecae293bb237d4f5a4dce6cc3bc79f02"} Jul 28 17:25:02.902617 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.902584 2570 scope.go:117] "RemoveContainer" containerID="6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278" Jul 28 17:25:02.911531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.911515 2570 scope.go:117] "RemoveContainer" containerID="6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278" Jul 28 17:25:02.911799 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:25:02.911781 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278\": container with ID starting with 6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278 not found: ID does not exist" containerID="6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278" Jul 28 17:25:02.911887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.911809 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278"} err="failed to get container status \"6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278\": rpc error: code = NotFound desc = could not find container \"6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278\": container with ID starting with 6739be2f4141f426c47fff4a424a0629ccd5b9987079f6178de8a3c078ee2278 not found: ID does not exist" Jul 28 17:25:02.924559 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.924527 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-95mnn"] Jul 28 17:25:02.928109 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:02.928085 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-95mnn"] Jul 28 17:25:03.724510 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:03.723759 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7330d427-24eb-401e-a40e-1fb9765e90f5" path="/var/lib/kubelet/pods/7330d427-24eb-401e-a40e-1fb9765e90f5/volumes" Jul 28 17:25:09.539597 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:09.539551 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 28 17:25:19.540038 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:19.539992 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 28 17:25:29.539707 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:29.539663 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" probeResult="failure" output="Get \"https://10.132.0.33:6567/health\": dial tcp 10.132.0.33:6567: connect: connection refused" Jul 28 17:25:29.540126 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:29.539850 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:25:32.041311 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.041282 2570 generic.go:356] "Generic (PLEG): container finished" podID="586d1507-c8ff-4abc-88a7-639139252e78" containerID="e71f33b88c624ed8e3cc82fd66b1ac05a631ebb02150bc058940c446382d2621" exitCode=137 Jul 28 17:25:32.041311 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.041305 2570 generic.go:356] "Generic (PLEG): container finished" podID="586d1507-c8ff-4abc-88a7-639139252e78" containerID="bdde7e88c94ae5b03946ab52854f07b5303683df9d600b6404f8a3d679037ac7" exitCode=137 Jul 28 17:25:32.041747 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.041363 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerDied","Data":"e71f33b88c624ed8e3cc82fd66b1ac05a631ebb02150bc058940c446382d2621"} Jul 28 17:25:32.041747 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.041416 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerDied","Data":"bdde7e88c94ae5b03946ab52854f07b5303683df9d600b6404f8a3d679037ac7"} Jul 28 17:25:32.498866 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.498823 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:25:32.630794 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.630690 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls\") pod \"586d1507-c8ff-4abc-88a7-639139252e78\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " Jul 28 17:25:32.630794 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.630772 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsbch\" (UniqueName: \"kubernetes.io/projected/586d1507-c8ff-4abc-88a7-639139252e78-kube-api-access-qsbch\") pod \"586d1507-c8ff-4abc-88a7-639139252e78\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " Jul 28 17:25:32.631130 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.630802 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/586d1507-c8ff-4abc-88a7-639139252e78-feast-data\") pod \"586d1507-c8ff-4abc-88a7-639139252e78\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " Jul 28 17:25:32.631130 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.630874 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls\") pod \"586d1507-c8ff-4abc-88a7-639139252e78\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " Jul 28 17:25:32.631130 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.630908 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls\") pod \"586d1507-c8ff-4abc-88a7-639139252e78\" (UID: \"586d1507-c8ff-4abc-88a7-639139252e78\") " Jul 28 17:25:32.633046 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.633009 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "586d1507-c8ff-4abc-88a7-639139252e78" (UID: "586d1507-c8ff-4abc-88a7-639139252e78"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:25:32.633187 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.633041 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls" (OuterVolumeSpecName: "online-tls") pod "586d1507-c8ff-4abc-88a7-639139252e78" (UID: "586d1507-c8ff-4abc-88a7-639139252e78"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:25:32.633255 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.633241 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "586d1507-c8ff-4abc-88a7-639139252e78" (UID: "586d1507-c8ff-4abc-88a7-639139252e78"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:25:32.633332 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.633314 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586d1507-c8ff-4abc-88a7-639139252e78-kube-api-access-qsbch" (OuterVolumeSpecName: "kube-api-access-qsbch") pod "586d1507-c8ff-4abc-88a7-639139252e78" (UID: "586d1507-c8ff-4abc-88a7-639139252e78"). InnerVolumeSpecName "kube-api-access-qsbch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:25:32.639731 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.639703 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/586d1507-c8ff-4abc-88a7-639139252e78-feast-data" (OuterVolumeSpecName: "feast-data") pod "586d1507-c8ff-4abc-88a7-639139252e78" (UID: "586d1507-c8ff-4abc-88a7-639139252e78"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:25:32.731743 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.731705 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsbch\" (UniqueName: \"kubernetes.io/projected/586d1507-c8ff-4abc-88a7-639139252e78-kube-api-access-qsbch\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:25:32.731743 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.731734 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/586d1507-c8ff-4abc-88a7-639139252e78-feast-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:25:32.731743 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.731743 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-online-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:25:32.731743 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.731750 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-registry-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:25:32.731743 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:32.731758 2570 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/586d1507-c8ff-4abc-88a7-639139252e78-offline-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:25:33.049708 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.049623 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" Jul 28 17:25:33.049708 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.049631 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-58d77f89-zhp9z" event={"ID":"586d1507-c8ff-4abc-88a7-639139252e78","Type":"ContainerDied","Data":"d5973f9926ba7c77937ca322bb1dfaae79cb6d2cd2d56d012b98630b4c1680b6"} Jul 28 17:25:33.049708 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.049703 2570 scope.go:117] "RemoveContainer" containerID="e71f33b88c624ed8e3cc82fd66b1ac05a631ebb02150bc058940c446382d2621" Jul 28 17:25:33.059234 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.059198 2570 scope.go:117] "RemoveContainer" containerID="546b4d34704042997ef7a88b4d8c4d7b6d3a79ab99bb3ca551ff8b63e170c982" Jul 28 17:25:33.068261 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.068239 2570 scope.go:117] "RemoveContainer" containerID="bdde7e88c94ae5b03946ab52854f07b5303683df9d600b6404f8a3d679037ac7" Jul 28 17:25:33.074005 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.073968 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-58d77f89-zhp9z"] Jul 28 17:25:33.081912 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.081863 2570 scope.go:117] "RemoveContainer" containerID="2649612b5687ee1cb0feede97e904297a2a8d851503857518700623dca1c1093" Jul 28 17:25:33.082317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.082262 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-58d77f89-zhp9z"] Jul 28 17:25:33.092177 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.092143 2570 scope.go:117] "RemoveContainer" containerID="89fc0d1044a667ca8c94ac898c3e8baa77b514d4baf14c12f31e09a560bdd8df" Jul 28 17:25:33.723151 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:25:33.723117 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="586d1507-c8ff-4abc-88a7-639139252e78" path="/var/lib/kubelet/pods/586d1507-c8ff-4abc-88a7-639139252e78/volumes" Jul 28 17:27:37.720424 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:37.720396 2570 scope.go:117] "RemoveContainer" containerID="0a4f63d3c7f36e8bc655c7e7d5c19a24e10540385e5dc738eff75c1c5e006fd2" Jul 28 17:27:37.729605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:37.729579 2570 scope.go:117] "RemoveContainer" containerID="5545b35d6b125ca8dfc3405a3e61611c33c843728617cd7318c8d6b69a50c355" Jul 28 17:27:37.738174 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:37.738003 2570 scope.go:117] "RemoveContainer" containerID="49e004e57abbaee8c88b31c4e547600b35da748982e3fe6e15e2c8bc692547c7" Jul 28 17:27:37.746196 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:37.746178 2570 scope.go:117] "RemoveContainer" containerID="1f41fe4cbb941314939f77307fb79074336ae6577f7e5f3d6c71334c72ed8264" Jul 28 17:27:37.753496 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:37.753478 2570 scope.go:117] "RemoveContainer" containerID="29aaf601ba6a8d38d803b185a9cd8d9548f71645bb308479c4ae1e8815634ac5" Jul 28 17:27:42.170590 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.170556 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bxp28"] Jul 28 17:27:42.171114 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171098 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="feast-init" Jul 28 17:27:42.171162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171119 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="feast-init" Jul 28 17:27:42.171162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171132 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" Jul 28 17:27:42.171162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171140 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" Jul 28 17:27:42.171162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171152 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7330d427-24eb-401e-a40e-1fb9765e90f5" containerName="postgres" Jul 28 17:27:42.171162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171160 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7330d427-24eb-401e-a40e-1fb9765e90f5" containerName="postgres" Jul 28 17:27:42.171317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171189 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="registry" Jul 28 17:27:42.171317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171197 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="registry" Jul 28 17:27:42.171317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171213 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="feast-apply" Jul 28 17:27:42.171317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171222 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="feast-apply" Jul 28 17:27:42.171317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171232 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="offline" Jul 28 17:27:42.171317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171240 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="offline" Jul 28 17:27:42.171317 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171314 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="online" Jul 28 17:27:42.171524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171327 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="registry" Jul 28 17:27:42.171524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171336 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="586d1507-c8ff-4abc-88a7-639139252e78" containerName="offline" Jul 28 17:27:42.171524 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.171349 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7330d427-24eb-401e-a40e-1fb9765e90f5" containerName="postgres" Jul 28 17:27:42.176298 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.176268 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" Jul 28 17:27:42.179206 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.179185 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jul 28 17:27:42.180616 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.180593 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-98zgm\"" Jul 28 17:27:42.180700 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.180669 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jul 28 17:27:42.194179 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.194149 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bxp28"] Jul 28 17:27:42.299434 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.299394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqgqg\" (UniqueName: \"kubernetes.io/projected/a365f4dc-0e30-4309-b6e6-1a4219ddb088-kube-api-access-cqgqg\") pod \"redis-69bbd798b6-bxp28\" (UID: \"a365f4dc-0e30-4309-b6e6-1a4219ddb088\") " pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" Jul 28 17:27:42.400039 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.399980 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqgqg\" (UniqueName: \"kubernetes.io/projected/a365f4dc-0e30-4309-b6e6-1a4219ddb088-kube-api-access-cqgqg\") pod \"redis-69bbd798b6-bxp28\" (UID: \"a365f4dc-0e30-4309-b6e6-1a4219ddb088\") " pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" Jul 28 17:27:42.411393 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.411361 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqgqg\" (UniqueName: \"kubernetes.io/projected/a365f4dc-0e30-4309-b6e6-1a4219ddb088-kube-api-access-cqgqg\") pod \"redis-69bbd798b6-bxp28\" (UID: \"a365f4dc-0e30-4309-b6e6-1a4219ddb088\") " pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" Jul 28 17:27:42.502646 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.502541 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" Jul 28 17:27:42.634943 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:42.634916 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bxp28"] Jul 28 17:27:42.637445 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:27:42.637417 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda365f4dc_0e30_4309_b6e6_1a4219ddb088.slice/crio-9266115884f16c07b06c4192cfd7544b6e54fe07dd9bb916f3b9da69b8ac0eaa WatchSource:0}: Error finding container 9266115884f16c07b06c4192cfd7544b6e54fe07dd9bb916f3b9da69b8ac0eaa: Status 404 returned error can't find the container with id 9266115884f16c07b06c4192cfd7544b6e54fe07dd9bb916f3b9da69b8ac0eaa Jul 28 17:27:43.602907 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:43.602867 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" event={"ID":"a365f4dc-0e30-4309-b6e6-1a4219ddb088","Type":"ContainerStarted","Data":"9266115884f16c07b06c4192cfd7544b6e54fe07dd9bb916f3b9da69b8ac0eaa"} Jul 28 17:27:44.608262 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:44.608230 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" event={"ID":"a365f4dc-0e30-4309-b6e6-1a4219ddb088","Type":"ContainerStarted","Data":"1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee"} Jul 28 17:27:44.626006 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:44.625951 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" podStartSLOduration=1.007243375 podStartE2EDuration="2.625936319s" totalImagesPullingTime="1.618692944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:44.624847919 +0000 UTC m=+1147.383708825" watchObservedRunningTime="2026-07-28 17:27:44.625936319 +0000 UTC m=+1147.384797232" Jul 28 17:27:47.668522 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.668488 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-hl7t8"] Jul 28 17:27:47.671913 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.671891 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:47.675163 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.675141 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Jul 28 17:27:47.680238 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.680215 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-hl7t8"] Jul 28 17:27:47.850714 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.850674 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts8qh\" (UniqueName: \"kubernetes.io/projected/ab8ac686-7fd7-4056-bafb-b2346d345114-kube-api-access-ts8qh\") pod \"postgres-864846d4f5-hl7t8\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:47.850915 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.850806 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ab8ac686-7fd7-4056-bafb-b2346d345114-postgresdata\") pod \"postgres-864846d4f5-hl7t8\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:47.956878 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.956756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ab8ac686-7fd7-4056-bafb-b2346d345114-postgresdata\") pod \"postgres-864846d4f5-hl7t8\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:47.957046 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.956803 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts8qh\" (UniqueName: \"kubernetes.io/projected/ab8ac686-7fd7-4056-bafb-b2346d345114-kube-api-access-ts8qh\") pod \"postgres-864846d4f5-hl7t8\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:47.957286 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.957266 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ab8ac686-7fd7-4056-bafb-b2346d345114-postgresdata\") pod \"postgres-864846d4f5-hl7t8\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:47.966578 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.966547 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts8qh\" (UniqueName: \"kubernetes.io/projected/ab8ac686-7fd7-4056-bafb-b2346d345114-kube-api-access-ts8qh\") pod \"postgres-864846d4f5-hl7t8\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:47.981392 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:47.981362 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:27:48.111066 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:48.111035 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-hl7t8"] Jul 28 17:27:48.114066 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:27:48.114038 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab8ac686_7fd7_4056_bafb_b2346d345114.slice/crio-ce372cb60e410ec4c45db07e17ea502e5c69098b4fb5deb3c467d398b9af4a5c WatchSource:0}: Error finding container ce372cb60e410ec4c45db07e17ea502e5c69098b4fb5deb3c467d398b9af4a5c: Status 404 returned error can't find the container with id ce372cb60e410ec4c45db07e17ea502e5c69098b4fb5deb3c467d398b9af4a5c Jul 28 17:27:48.628962 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:48.628925 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" event={"ID":"ab8ac686-7fd7-4056-bafb-b2346d345114","Type":"ContainerStarted","Data":"9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170"} Jul 28 17:27:48.628962 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:48.628969 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" event={"ID":"ab8ac686-7fd7-4056-bafb-b2346d345114","Type":"ContainerStarted","Data":"ce372cb60e410ec4c45db07e17ea502e5c69098b4fb5deb3c467d398b9af4a5c"} Jul 28 17:27:48.645846 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:48.645771 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" podStartSLOduration=1.645753716 podStartE2EDuration="1.645753716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:27:48.645385945 +0000 UTC m=+1151.404246861" watchObservedRunningTime="2026-07-28 17:27:48.645753716 +0000 UTC m=+1151.404614631" Jul 28 17:27:53.231719 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.231687 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg"] Jul 28 17:27:53.235850 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.235815 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.238771 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.238748 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Jul 28 17:27:53.238916 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.238794 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jul 28 17:27:53.238916 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.238755 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Jul 28 17:27:53.238916 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.238746 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-96lg8\"" Jul 28 17:27:53.247300 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.247276 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg"] Jul 28 17:27:53.401197 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.401166 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-registry-tls\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.401197 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.401200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.401425 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.401251 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws5db\" (UniqueName: \"kubernetes.io/projected/f1856775-389d-424a-bec8-ff8afb61b0e1-kube-api-access-ws5db\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.401425 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.401279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f1856775-389d-424a-bec8-ff8afb61b0e1-feast-data\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.502006 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.501919 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws5db\" (UniqueName: \"kubernetes.io/projected/f1856775-389d-424a-bec8-ff8afb61b0e1-kube-api-access-ws5db\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.502006 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.501988 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f1856775-389d-424a-bec8-ff8afb61b0e1-feast-data\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.502209 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.502036 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-registry-tls\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.502209 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.502069 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.502209 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:27:53.502203 2570 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Jul 28 17:27:53.502352 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:27:53.502272 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls podName:f1856775-389d-424a-bec8-ff8afb61b0e1 nodeName:}" failed. No retries permitted until 2026-07-28 17:27:54.002250471 +0000 UTC m=+1156.761111362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls") pod "feast-credit-scoring-84f65bb66f-njhvg" (UID: "f1856775-389d-424a-bec8-ff8afb61b0e1") : secret "feast-credit-scoring-online-tls" not found Jul 28 17:27:53.502658 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.502630 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f1856775-389d-424a-bec8-ff8afb61b0e1-feast-data\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.504648 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.504623 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-registry-tls\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:53.511404 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:53.511376 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws5db\" (UniqueName: \"kubernetes.io/projected/f1856775-389d-424a-bec8-ff8afb61b0e1-kube-api-access-ws5db\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:54.007452 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:54.007415 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:54.009790 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:54.009766 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls\") pod \"feast-credit-scoring-84f65bb66f-njhvg\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:54.145450 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:54.145397 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:27:54.286657 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:54.286632 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg"] Jul 28 17:27:54.287582 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:27:54.287562 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1856775_389d_424a_bec8_ff8afb61b0e1.slice/crio-4606c054654186070aa8fd2747a6141a22f911004b1fb8715b645dccf33c4869 WatchSource:0}: Error finding container 4606c054654186070aa8fd2747a6141a22f911004b1fb8715b645dccf33c4869: Status 404 returned error can't find the container with id 4606c054654186070aa8fd2747a6141a22f911004b1fb8715b645dccf33c4869 Jul 28 17:27:54.657016 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:54.656975 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerStarted","Data":"4890e50cee0c60bfbd14b929c75f15713a06952244a0716c5f31879b2730a6c3"} Jul 28 17:27:54.657191 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:54.657031 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerStarted","Data":"4606c054654186070aa8fd2747a6141a22f911004b1fb8715b645dccf33c4869"} Jul 28 17:27:56.667227 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:56.667193 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerID="4890e50cee0c60bfbd14b929c75f15713a06952244a0716c5f31879b2730a6c3" exitCode=0 Jul 28 17:27:56.667621 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:56.667268 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerDied","Data":"4890e50cee0c60bfbd14b929c75f15713a06952244a0716c5f31879b2730a6c3"} Jul 28 17:27:57.674495 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:27:57.674461 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerStarted","Data":"1967ce3d2a3fe76e0302bdaaa60ae5e9fafdd9979bee7354b868708dfed8c04e"} Jul 28 17:28:04.707522 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:04.707485 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerID="1967ce3d2a3fe76e0302bdaaa60ae5e9fafdd9979bee7354b868708dfed8c04e" exitCode=0 Jul 28 17:28:04.708062 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:04.707560 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerDied","Data":"1967ce3d2a3fe76e0302bdaaa60ae5e9fafdd9979bee7354b868708dfed8c04e"} Jul 28 17:28:05.716331 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:05.716287 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerStarted","Data":"cdf669c14a7fbd12251b838ebf29075a73961df9e5b12540e9d51fe14f6e41a9"} Jul 28 17:28:05.716704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:05.716410 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerStarted","Data":"192fb160192a80a21cb9716a99c8c1d3c1e175c096708476504b0978dad63875"} Jul 28 17:28:05.751184 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:05.751130 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podStartSLOduration=12.751114273 podStartE2EDuration="12.751114273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:28:05.749039224 +0000 UTC m=+1168.507900138" watchObservedRunningTime="2026-07-28 17:28:05.751114273 +0000 UTC m=+1168.509975187" Jul 28 17:28:06.146168 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:06.146119 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:06.146358 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:06.146196 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:06.147565 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:06.147522 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Jul 28 17:28:06.147565 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:06.147552 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 28 17:28:09.146315 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:09.146266 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Jul 28 17:28:09.146738 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:09.146284 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 28 17:28:12.146260 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:12.146206 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Jul 28 17:28:12.151979 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:12.151946 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:12.152468 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:12.152441 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:12.155576 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:12.155549 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:14.146401 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:14.146361 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:15.146271 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:15.146225 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.36:6571: connect: connection refused" Jul 28 17:28:18.146484 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:18.146448 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:18.147246 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:18.147227 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:28:19.164629 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.164588 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8"] Jul 28 17:28:19.168842 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.168802 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.171605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.171577 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jul 28 17:28:19.171605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.171592 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-jxvgn\"" Jul 28 17:28:19.171605 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.171578 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jul 28 17:28:19.171959 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.171943 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jul 28 17:28:19.179162 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.179136 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8"] Jul 28 17:28:19.226860 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.226802 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.227009 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.226922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96ef078b-6034-4935-a905-457bb7b77445-feast-data\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.227086 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.227020 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbxbt\" (UniqueName: \"kubernetes.io/projected/96ef078b-6034-4935-a905-457bb7b77445-kube-api-access-wbxbt\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.227138 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.227080 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-registry-tls\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.328275 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.328237 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96ef078b-6034-4935-a905-457bb7b77445-feast-data\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.328470 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.328304 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbxbt\" (UniqueName: \"kubernetes.io/projected/96ef078b-6034-4935-a905-457bb7b77445-kube-api-access-wbxbt\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.328470 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.328330 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-registry-tls\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.328470 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.328361 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.328470 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:28:19.328459 2570 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-online-tls: secret "feast-driver-ranking-online-tls" not found Jul 28 17:28:19.328641 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:28:19.328515 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls podName:96ef078b-6034-4935-a905-457bb7b77445 nodeName:}" failed. No retries permitted until 2026-07-28 17:28:19.828498693 +0000 UTC m=+1182.587359585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls") pod "feast-driver-ranking-6975d78fcd-ct5j8" (UID: "96ef078b-6034-4935-a905-457bb7b77445") : secret "feast-driver-ranking-online-tls" not found Jul 28 17:28:19.328689 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.328650 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96ef078b-6034-4935-a905-457bb7b77445-feast-data\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.330786 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.330765 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-registry-tls\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.342898 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.342869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbxbt\" (UniqueName: \"kubernetes.io/projected/96ef078b-6034-4935-a905-457bb7b77445-kube-api-access-wbxbt\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.832984 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.832950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:19.835409 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:19.835384 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls\") pod \"feast-driver-ranking-6975d78fcd-ct5j8\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:20.079106 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:20.079070 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:20.218880 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:20.218854 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8"] Jul 28 17:28:20.220902 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:28:20.220877 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96ef078b_6034_4935_a905_457bb7b77445.slice/crio-4d63eb871492817d4f1288b869cd9e240aa6daffcddf7846b3375de7627d31ff WatchSource:0}: Error finding container 4d63eb871492817d4f1288b869cd9e240aa6daffcddf7846b3375de7627d31ff: Status 404 returned error can't find the container with id 4d63eb871492817d4f1288b869cd9e240aa6daffcddf7846b3375de7627d31ff Jul 28 17:28:20.794661 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:20.794623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerStarted","Data":"8e73bb2261524aca484117faa82ec69141a4a043a4a06d47d2162bfbf26e33ce"} Jul 28 17:28:20.794942 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:20.794686 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerStarted","Data":"4d63eb871492817d4f1288b869cd9e240aa6daffcddf7846b3375de7627d31ff"} Jul 28 17:28:25.819209 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:25.819172 2570 generic.go:356] "Generic (PLEG): container finished" podID="96ef078b-6034-4935-a905-457bb7b77445" containerID="8e73bb2261524aca484117faa82ec69141a4a043a4a06d47d2162bfbf26e33ce" exitCode=0 Jul 28 17:28:25.819209 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:25.819213 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerDied","Data":"8e73bb2261524aca484117faa82ec69141a4a043a4a06d47d2162bfbf26e33ce"} Jul 28 17:28:26.826756 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:26.826720 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerStarted","Data":"f2fe97b4264e73d8c1765fd8a70a0c90a95b142ee89caeade7ba450b7afe4327"} Jul 28 17:28:32.858086 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:32.858049 2570 generic.go:356] "Generic (PLEG): container finished" podID="96ef078b-6034-4935-a905-457bb7b77445" containerID="f2fe97b4264e73d8c1765fd8a70a0c90a95b142ee89caeade7ba450b7afe4327" exitCode=0 Jul 28 17:28:32.858456 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:32.858128 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerDied","Data":"f2fe97b4264e73d8c1765fd8a70a0c90a95b142ee89caeade7ba450b7afe4327"} Jul 28 17:28:33.868395 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:33.868356 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerStarted","Data":"8a536f6dd8ae8255d704fdfc794cef7dc36ba858a13a5b795a9cf9ae50c1baeb"} Jul 28 17:28:33.868901 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:33.868414 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerStarted","Data":"79457370ee4d5751169fc97d92240d5d6ebb1acea45dba732a1f4afba973e274"} Jul 28 17:28:33.897308 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:33.897247 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podStartSLOduration=14.897229512 podStartE2EDuration="14.897229512s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:28:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:28:33.895911036 +0000 UTC m=+1196.654771950" watchObservedRunningTime="2026-07-28 17:28:33.897229512 +0000 UTC m=+1196.656090425" Jul 28 17:28:35.079500 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:35.079450 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:35.080031 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:35.079534 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:35.080730 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:35.080698 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.37:6571: connect: connection refused" Jul 28 17:28:35.080850 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:35.080741 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 28 17:28:38.079473 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:38.079423 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.37:6571: connect: connection refused" Jul 28 17:28:38.079900 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:38.079663 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 28 17:28:40.079725 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:40.079680 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:40.080251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:40.079769 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:41.080340 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:41.080284 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.37:6571: connect: connection refused" Jul 28 17:28:41.086357 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:41.086329 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:41.089949 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:41.089920 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:44.079705 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:44.079656 2570 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" probeResult="failure" output="dial tcp 10.132.0.37:6571: connect: connection refused" Jul 28 17:28:47.080082 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:47.080041 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:28:47.080864 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:28:47.080845 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:30:04.824314 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:04.824279 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt"] Jul 28 17:30:04.824818 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:04.824492 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" podUID="096131fa-3f5c-4d3e-9738-533c1b0260b0" containerName="manager" containerID="cri-o://6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31" gracePeriod=10 Jul 28 17:30:05.099237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.099208 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:30:05.106895 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.106808 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg"] Jul 28 17:30:05.107154 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.107120 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" containerID="cri-o://cdf669c14a7fbd12251b838ebf29075a73961df9e5b12540e9d51fe14f6e41a9" gracePeriod=30 Jul 28 17:30:05.107267 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.107109 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" containerID="cri-o://192fb160192a80a21cb9716a99c8c1d3c1e175c096708476504b0978dad63875" gracePeriod=30 Jul 28 17:30:05.109700 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.109663 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8"] Jul 28 17:30:05.109988 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.109934 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" containerID="cri-o://79457370ee4d5751169fc97d92240d5d6ebb1acea45dba732a1f4afba973e274" gracePeriod=30 Jul 28 17:30:05.109988 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.109966 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" containerID="cri-o://8a536f6dd8ae8255d704fdfc794cef7dc36ba858a13a5b795a9cf9ae50c1baeb" gracePeriod=30 Jul 28 17:30:05.153669 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.153639 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh84p\" (UniqueName: \"kubernetes.io/projected/096131fa-3f5c-4d3e-9738-533c1b0260b0-kube-api-access-nh84p\") pod \"096131fa-3f5c-4d3e-9738-533c1b0260b0\" (UID: \"096131fa-3f5c-4d3e-9738-533c1b0260b0\") " Jul 28 17:30:05.155772 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.155744 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096131fa-3f5c-4d3e-9738-533c1b0260b0-kube-api-access-nh84p" (OuterVolumeSpecName: "kube-api-access-nh84p") pod "096131fa-3f5c-4d3e-9738-533c1b0260b0" (UID: "096131fa-3f5c-4d3e-9738-533c1b0260b0"). InnerVolumeSpecName "kube-api-access-nh84p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:30:05.254595 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.254559 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh84p\" (UniqueName: \"kubernetes.io/projected/096131fa-3f5c-4d3e-9738-533c1b0260b0-kube-api-access-nh84p\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:05.303147 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.303105 2570 generic.go:356] "Generic (PLEG): container finished" podID="096131fa-3f5c-4d3e-9738-533c1b0260b0" containerID="6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31" exitCode=0 Jul 28 17:30:05.303147 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.303147 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" Jul 28 17:30:05.303342 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.303174 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" event={"ID":"096131fa-3f5c-4d3e-9738-533c1b0260b0","Type":"ContainerDied","Data":"6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31"} Jul 28 17:30:05.303342 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.303227 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt" event={"ID":"096131fa-3f5c-4d3e-9738-533c1b0260b0","Type":"ContainerDied","Data":"15656deeedadd423dccebdb8e78c1f368e63ecb646d9edd86f2dbf0c30609da8"} Jul 28 17:30:05.303342 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.303256 2570 scope.go:117] "RemoveContainer" containerID="6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31" Jul 28 17:30:05.312506 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.312488 2570 scope.go:117] "RemoveContainer" containerID="6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31" Jul 28 17:30:05.312764 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:30:05.312740 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31\": container with ID starting with 6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31 not found: ID does not exist" containerID="6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31" Jul 28 17:30:05.312811 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.312776 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31"} err="failed to get container status \"6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31\": rpc error: code = NotFound desc = could not find container \"6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31\": container with ID starting with 6912275f94605589e1e5583554532317595a7926597ac03c1d0c9d463e354a31 not found: ID does not exist" Jul 28 17:30:05.326125 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.326044 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt"] Jul 28 17:30:05.330430 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.330392 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-7b7db985fb-zhntt"] Jul 28 17:30:05.724630 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:05.724594 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="096131fa-3f5c-4d3e-9738-533c1b0260b0" path="/var/lib/kubelet/pods/096131fa-3f5c-4d3e-9738-533c1b0260b0/volumes" Jul 28 17:30:06.311321 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:06.311287 2570 generic.go:356] "Generic (PLEG): container finished" podID="96ef078b-6034-4935-a905-457bb7b77445" containerID="8a536f6dd8ae8255d704fdfc794cef7dc36ba858a13a5b795a9cf9ae50c1baeb" exitCode=0 Jul 28 17:30:06.311709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:06.311358 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerDied","Data":"8a536f6dd8ae8255d704fdfc794cef7dc36ba858a13a5b795a9cf9ae50c1baeb"} Jul 28 17:30:06.314488 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:06.314462 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerID="cdf669c14a7fbd12251b838ebf29075a73961df9e5b12540e9d51fe14f6e41a9" exitCode=0 Jul 28 17:30:06.314616 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:06.314499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerDied","Data":"cdf669c14a7fbd12251b838ebf29075a73961df9e5b12540e9d51fe14f6e41a9"} Jul 28 17:30:07.262682 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.262647 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-hl7t8"] Jul 28 17:30:07.262885 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.262842 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" podUID="ab8ac686-7fd7-4056-bafb-b2346d345114" containerName="postgres" containerID="cri-o://9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170" gracePeriod=30 Jul 28 17:30:07.270848 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.270130 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bxp28"] Jul 28 17:30:07.270848 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.270326 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" podUID="a365f4dc-0e30-4309-b6e6-1a4219ddb088" containerName="redis" containerID="cri-o://1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee" gracePeriod=30 Jul 28 17:30:07.532118 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.532091 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:30:07.535035 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.535014 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" Jul 28 17:30:07.583130 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.583096 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ts8qh\" (UniqueName: \"kubernetes.io/projected/ab8ac686-7fd7-4056-bafb-b2346d345114-kube-api-access-ts8qh\") pod \"ab8ac686-7fd7-4056-bafb-b2346d345114\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " Jul 28 17:30:07.583326 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.583140 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ab8ac686-7fd7-4056-bafb-b2346d345114-postgresdata\") pod \"ab8ac686-7fd7-4056-bafb-b2346d345114\" (UID: \"ab8ac686-7fd7-4056-bafb-b2346d345114\") " Jul 28 17:30:07.583326 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.583208 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cqgqg\" (UniqueName: \"kubernetes.io/projected/a365f4dc-0e30-4309-b6e6-1a4219ddb088-kube-api-access-cqgqg\") pod \"a365f4dc-0e30-4309-b6e6-1a4219ddb088\" (UID: \"a365f4dc-0e30-4309-b6e6-1a4219ddb088\") " Jul 28 17:30:07.585415 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.585385 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a365f4dc-0e30-4309-b6e6-1a4219ddb088-kube-api-access-cqgqg" (OuterVolumeSpecName: "kube-api-access-cqgqg") pod "a365f4dc-0e30-4309-b6e6-1a4219ddb088" (UID: "a365f4dc-0e30-4309-b6e6-1a4219ddb088"). InnerVolumeSpecName "kube-api-access-cqgqg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:30:07.585644 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.585619 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8ac686-7fd7-4056-bafb-b2346d345114-kube-api-access-ts8qh" (OuterVolumeSpecName: "kube-api-access-ts8qh") pod "ab8ac686-7fd7-4056-bafb-b2346d345114" (UID: "ab8ac686-7fd7-4056-bafb-b2346d345114"). InnerVolumeSpecName "kube-api-access-ts8qh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:30:07.613065 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.613000 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8ac686-7fd7-4056-bafb-b2346d345114-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "ab8ac686-7fd7-4056-bafb-b2346d345114" (UID: "ab8ac686-7fd7-4056-bafb-b2346d345114"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:30:07.684320 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.684282 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqgqg\" (UniqueName: \"kubernetes.io/projected/a365f4dc-0e30-4309-b6e6-1a4219ddb088-kube-api-access-cqgqg\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:07.684320 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.684315 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts8qh\" (UniqueName: \"kubernetes.io/projected/ab8ac686-7fd7-4056-bafb-b2346d345114-kube-api-access-ts8qh\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:07.684320 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:07.684325 2570 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ab8ac686-7fd7-4056-bafb-b2346d345114-postgresdata\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:08.325770 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.325742 2570 generic.go:356] "Generic (PLEG): container finished" podID="a365f4dc-0e30-4309-b6e6-1a4219ddb088" containerID="1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee" exitCode=0 Jul 28 17:30:08.325950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.325780 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" Jul 28 17:30:08.325950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.325844 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" event={"ID":"a365f4dc-0e30-4309-b6e6-1a4219ddb088","Type":"ContainerDied","Data":"1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee"} Jul 28 17:30:08.325950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.325892 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-bxp28" event={"ID":"a365f4dc-0e30-4309-b6e6-1a4219ddb088","Type":"ContainerDied","Data":"9266115884f16c07b06c4192cfd7544b6e54fe07dd9bb916f3b9da69b8ac0eaa"} Jul 28 17:30:08.325950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.325916 2570 scope.go:117] "RemoveContainer" containerID="1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee" Jul 28 17:30:08.327237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.327133 2570 generic.go:356] "Generic (PLEG): container finished" podID="ab8ac686-7fd7-4056-bafb-b2346d345114" containerID="9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170" exitCode=0 Jul 28 17:30:08.327237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.327161 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" event={"ID":"ab8ac686-7fd7-4056-bafb-b2346d345114","Type":"ContainerDied","Data":"9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170"} Jul 28 17:30:08.327237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.327172 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" Jul 28 17:30:08.327237 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.327187 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-hl7t8" event={"ID":"ab8ac686-7fd7-4056-bafb-b2346d345114","Type":"ContainerDied","Data":"ce372cb60e410ec4c45db07e17ea502e5c69098b4fb5deb3c467d398b9af4a5c"} Jul 28 17:30:08.334347 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.334331 2570 scope.go:117] "RemoveContainer" containerID="1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee" Jul 28 17:30:08.334582 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:30:08.334565 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee\": container with ID starting with 1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee not found: ID does not exist" containerID="1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee" Jul 28 17:30:08.334634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.334590 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee"} err="failed to get container status \"1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee\": rpc error: code = NotFound desc = could not find container \"1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee\": container with ID starting with 1292b69c9d0cc88b459fd2cea88f18f5d7b3bbedf2461a399264216fe56a69ee not found: ID does not exist" Jul 28 17:30:08.334634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.334609 2570 scope.go:117] "RemoveContainer" containerID="9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170" Jul 28 17:30:08.343951 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.343930 2570 scope.go:117] "RemoveContainer" containerID="9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170" Jul 28 17:30:08.344226 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:30:08.344199 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170\": container with ID starting with 9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170 not found: ID does not exist" containerID="9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170" Jul 28 17:30:08.344297 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.344234 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170"} err="failed to get container status \"9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170\": rpc error: code = NotFound desc = could not find container \"9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170\": container with ID starting with 9e42513dca9befc3f89e0f66ac4206e517f8b429ac979a8de51570071ae35170 not found: ID does not exist" Jul 28 17:30:08.344935 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.344913 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-hl7t8"] Jul 28 17:30:08.347257 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.347237 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-hl7t8"] Jul 28 17:30:08.357634 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.357612 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bxp28"] Jul 28 17:30:08.361358 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:08.361335 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-bxp28"] Jul 28 17:30:09.723310 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:09.723273 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a365f4dc-0e30-4309-b6e6-1a4219ddb088" path="/var/lib/kubelet/pods/a365f4dc-0e30-4309-b6e6-1a4219ddb088/volumes" Jul 28 17:30:09.723674 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:09.723634 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab8ac686-7fd7-4056-bafb-b2346d345114" path="/var/lib/kubelet/pods/ab8ac686-7fd7-4056-bafb-b2346d345114/volumes" Jul 28 17:30:11.087030 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:11.086989 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 28 17:30:12.153213 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:12.153163 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 28 17:30:21.087295 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:21.087246 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 28 17:30:22.153180 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:22.153134 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 28 17:30:31.087783 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:31.087735 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" probeResult="failure" output="Get \"https://10.132.0.37:6567/health\": dial tcp 10.132.0.37:6567: connect: connection refused" Jul 28 17:30:31.088209 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:31.087894 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:30:32.153535 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:32.153491 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" probeResult="failure" output="Get \"https://10.132.0.36:6567/health\": dial tcp 10.132.0.36:6567: connect: connection refused" Jul 28 17:30:32.153950 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:32.153614 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:30:35.450450 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.450356 2570 generic.go:356] "Generic (PLEG): container finished" podID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerID="192fb160192a80a21cb9716a99c8c1d3c1e175c096708476504b0978dad63875" exitCode=137 Jul 28 17:30:35.450450 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.450436 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerDied","Data":"192fb160192a80a21cb9716a99c8c1d3c1e175c096708476504b0978dad63875"} Jul 28 17:30:35.452742 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.452722 2570 generic.go:356] "Generic (PLEG): container finished" podID="96ef078b-6034-4935-a905-457bb7b77445" containerID="79457370ee4d5751169fc97d92240d5d6ebb1acea45dba732a1f4afba973e274" exitCode=137 Jul 28 17:30:35.452849 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.452754 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerDied","Data":"79457370ee4d5751169fc97d92240d5d6ebb1acea45dba732a1f4afba973e274"} Jul 28 17:30:35.787087 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.787051 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:30:35.790288 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.790268 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:30:35.824862 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.824822 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws5db\" (UniqueName: \"kubernetes.io/projected/f1856775-389d-424a-bec8-ff8afb61b0e1-kube-api-access-ws5db\") pod \"f1856775-389d-424a-bec8-ff8afb61b0e1\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " Jul 28 17:30:35.825054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.824873 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbxbt\" (UniqueName: \"kubernetes.io/projected/96ef078b-6034-4935-a905-457bb7b77445-kube-api-access-wbxbt\") pod \"96ef078b-6034-4935-a905-457bb7b77445\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " Jul 28 17:30:35.825054 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.824922 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-registry-tls\") pod \"96ef078b-6034-4935-a905-457bb7b77445\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " Jul 28 17:30:35.825190 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.825064 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96ef078b-6034-4935-a905-457bb7b77445-feast-data\") pod \"96ef078b-6034-4935-a905-457bb7b77445\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " Jul 28 17:30:35.825190 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.825125 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-registry-tls\") pod \"f1856775-389d-424a-bec8-ff8afb61b0e1\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " Jul 28 17:30:35.825190 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.825152 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls\") pod \"96ef078b-6034-4935-a905-457bb7b77445\" (UID: \"96ef078b-6034-4935-a905-457bb7b77445\") " Jul 28 17:30:35.825190 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.825184 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls\") pod \"f1856775-389d-424a-bec8-ff8afb61b0e1\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " Jul 28 17:30:35.825349 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.825214 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f1856775-389d-424a-bec8-ff8afb61b0e1-feast-data\") pod \"f1856775-389d-424a-bec8-ff8afb61b0e1\" (UID: \"f1856775-389d-424a-bec8-ff8afb61b0e1\") " Jul 28 17:30:35.825573 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.825528 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96ef078b-6034-4935-a905-457bb7b77445-feast-data" (OuterVolumeSpecName: "feast-data") pod "96ef078b-6034-4935-a905-457bb7b77445" (UID: "96ef078b-6034-4935-a905-457bb7b77445"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:30:35.827673 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.827643 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1856775-389d-424a-bec8-ff8afb61b0e1-kube-api-access-ws5db" (OuterVolumeSpecName: "kube-api-access-ws5db") pod "f1856775-389d-424a-bec8-ff8afb61b0e1" (UID: "f1856775-389d-424a-bec8-ff8afb61b0e1"). InnerVolumeSpecName "kube-api-access-ws5db". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:30:35.827788 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.827641 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f1856775-389d-424a-bec8-ff8afb61b0e1" (UID: "f1856775-389d-424a-bec8-ff8afb61b0e1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:30:35.828008 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.827980 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls" (OuterVolumeSpecName: "online-tls") pod "f1856775-389d-424a-bec8-ff8afb61b0e1" (UID: "f1856775-389d-424a-bec8-ff8afb61b0e1"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:30:35.828008 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.827998 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "96ef078b-6034-4935-a905-457bb7b77445" (UID: "96ef078b-6034-4935-a905-457bb7b77445"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:30:35.828242 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.828220 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls" (OuterVolumeSpecName: "online-tls") pod "96ef078b-6034-4935-a905-457bb7b77445" (UID: "96ef078b-6034-4935-a905-457bb7b77445"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:30:35.828319 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.828220 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96ef078b-6034-4935-a905-457bb7b77445-kube-api-access-wbxbt" (OuterVolumeSpecName: "kube-api-access-wbxbt") pod "96ef078b-6034-4935-a905-457bb7b77445" (UID: "96ef078b-6034-4935-a905-457bb7b77445"). InnerVolumeSpecName "kube-api-access-wbxbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:30:35.836284 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.836261 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f1856775-389d-424a-bec8-ff8afb61b0e1-feast-data" (OuterVolumeSpecName: "feast-data") pod "f1856775-389d-424a-bec8-ff8afb61b0e1" (UID: "f1856775-389d-424a-bec8-ff8afb61b0e1"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:30:35.926350 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926319 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws5db\" (UniqueName: \"kubernetes.io/projected/f1856775-389d-424a-bec8-ff8afb61b0e1-kube-api-access-ws5db\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:35.926350 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926347 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbxbt\" (UniqueName: \"kubernetes.io/projected/96ef078b-6034-4935-a905-457bb7b77445-kube-api-access-wbxbt\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:35.926350 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926359 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-registry-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:35.926582 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926367 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96ef078b-6034-4935-a905-457bb7b77445-feast-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:35.926582 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926374 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-registry-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:35.926582 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926381 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96ef078b-6034-4935-a905-457bb7b77445-online-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:35.926582 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926389 2570 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/f1856775-389d-424a-bec8-ff8afb61b0e1-online-tls\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:35.926582 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:35.926396 2570 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/f1856775-389d-424a-bec8-ff8afb61b0e1-feast-data\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:30:36.459794 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.459768 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" Jul 28 17:30:36.460293 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.459786 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8" event={"ID":"96ef078b-6034-4935-a905-457bb7b77445","Type":"ContainerDied","Data":"4d63eb871492817d4f1288b869cd9e240aa6daffcddf7846b3375de7627d31ff"} Jul 28 17:30:36.460293 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.459867 2570 scope.go:117] "RemoveContainer" containerID="8a536f6dd8ae8255d704fdfc794cef7dc36ba858a13a5b795a9cf9ae50c1baeb" Jul 28 17:30:36.464974 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.464941 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" event={"ID":"f1856775-389d-424a-bec8-ff8afb61b0e1","Type":"ContainerDied","Data":"4606c054654186070aa8fd2747a6141a22f911004b1fb8715b645dccf33c4869"} Jul 28 17:30:36.465087 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.464956 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg" Jul 28 17:30:36.471424 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.471400 2570 scope.go:117] "RemoveContainer" containerID="79457370ee4d5751169fc97d92240d5d6ebb1acea45dba732a1f4afba973e274" Jul 28 17:30:36.480064 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.480042 2570 scope.go:117] "RemoveContainer" containerID="f2fe97b4264e73d8c1765fd8a70a0c90a95b142ee89caeade7ba450b7afe4327" Jul 28 17:30:36.485095 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.485069 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8"] Jul 28 17:30:36.489222 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.489192 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-6975d78fcd-ct5j8"] Jul 28 17:30:36.489620 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.489597 2570 scope.go:117] "RemoveContainer" containerID="8e73bb2261524aca484117faa82ec69141a4a043a4a06d47d2162bfbf26e33ce" Jul 28 17:30:36.497531 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.497508 2570 scope.go:117] "RemoveContainer" containerID="cdf669c14a7fbd12251b838ebf29075a73961df9e5b12540e9d51fe14f6e41a9" Jul 28 17:30:36.499299 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.499275 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg"] Jul 28 17:30:36.502769 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.502749 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-84f65bb66f-njhvg"] Jul 28 17:30:36.505961 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.505946 2570 scope.go:117] "RemoveContainer" containerID="192fb160192a80a21cb9716a99c8c1d3c1e175c096708476504b0978dad63875" Jul 28 17:30:36.514098 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.514077 2570 scope.go:117] "RemoveContainer" containerID="1967ce3d2a3fe76e0302bdaaa60ae5e9fafdd9979bee7354b868708dfed8c04e" Jul 28 17:30:36.521887 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:36.521857 2570 scope.go:117] "RemoveContainer" containerID="4890e50cee0c60bfbd14b929c75f15713a06952244a0716c5f31879b2730a6c3" Jul 28 17:30:37.727677 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:37.727644 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96ef078b-6034-4935-a905-457bb7b77445" path="/var/lib/kubelet/pods/96ef078b-6034-4935-a905-457bb7b77445/volumes" Jul 28 17:30:37.728147 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:30:37.728133 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" path="/var/lib/kubelet/pods/f1856775-389d-424a-bec8-ff8afb61b0e1/volumes" Jul 28 17:41:12.995798 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.995755 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fzqtb/must-gather-8rctm"] Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996095 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996112 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996125 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996134 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996151 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="feast-init" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996165 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="feast-init" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996175 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996184 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996197 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="096131fa-3f5c-4d3e-9738-533c1b0260b0" containerName="manager" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996205 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="096131fa-3f5c-4d3e-9738-533c1b0260b0" containerName="manager" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996212 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="feast-apply" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996218 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="feast-apply" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996225 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a365f4dc-0e30-4309-b6e6-1a4219ddb088" containerName="redis" Jul 28 17:41:12.996224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996231 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a365f4dc-0e30-4309-b6e6-1a4219ddb088" containerName="redis" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996239 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8ac686-7fd7-4056-bafb-b2346d345114" containerName="postgres" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996247 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8ac686-7fd7-4056-bafb-b2346d345114" containerName="postgres" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996261 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996266 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996277 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="feast-apply" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996282 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="feast-apply" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996289 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="feast-init" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996294 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="feast-init" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996364 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="online" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996372 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="registry" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996379 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="96ef078b-6034-4935-a905-457bb7b77445" containerName="online" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996385 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a365f4dc-0e30-4309-b6e6-1a4219ddb088" containerName="redis" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996390 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab8ac686-7fd7-4056-bafb-b2346d345114" containerName="postgres" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996396 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1856775-389d-424a-bec8-ff8afb61b0e1" containerName="registry" Jul 28 17:41:12.996704 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.996401 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="096131fa-3f5c-4d3e-9738-533c1b0260b0" containerName="manager" Jul 28 17:41:12.999399 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:12.999379 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.004285 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.004265 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fzqtb\"/\"default-dockercfg-vw6kp\"" Jul 28 17:41:13.004391 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.004271 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fzqtb\"/\"openshift-service-ca.crt\"" Jul 28 17:41:13.005449 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.005432 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fzqtb\"/\"kube-root-ca.crt\"" Jul 28 17:41:13.010527 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.010506 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fzqtb/must-gather-8rctm"] Jul 28 17:41:13.130090 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.130052 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b46555c-cea3-40c3-992f-8b966efbab8b-must-gather-output\") pod \"must-gather-8rctm\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.130273 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.130117 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtqkx\" (UniqueName: \"kubernetes.io/projected/4b46555c-cea3-40c3-992f-8b966efbab8b-kube-api-access-mtqkx\") pod \"must-gather-8rctm\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.231439 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.231400 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtqkx\" (UniqueName: \"kubernetes.io/projected/4b46555c-cea3-40c3-992f-8b966efbab8b-kube-api-access-mtqkx\") pod \"must-gather-8rctm\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.231588 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.231467 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b46555c-cea3-40c3-992f-8b966efbab8b-must-gather-output\") pod \"must-gather-8rctm\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.231763 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.231741 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b46555c-cea3-40c3-992f-8b966efbab8b-must-gather-output\") pod \"must-gather-8rctm\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.240310 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.240286 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtqkx\" (UniqueName: \"kubernetes.io/projected/4b46555c-cea3-40c3-992f-8b966efbab8b-kube-api-access-mtqkx\") pod \"must-gather-8rctm\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.319134 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.319047 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:13.447089 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.447064 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fzqtb/must-gather-8rctm"] Jul 28 17:41:13.449474 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:41:13.449447 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b46555c_cea3_40c3_992f_8b966efbab8b.slice/crio-9362be96ef2cafd049be21580637a254ff4dde4573a95c36aaa8f6d33cc02c3e WatchSource:0}: Error finding container 9362be96ef2cafd049be21580637a254ff4dde4573a95c36aaa8f6d33cc02c3e: Status 404 returned error can't find the container with id 9362be96ef2cafd049be21580637a254ff4dde4573a95c36aaa8f6d33cc02c3e Jul 28 17:41:13.451224 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:13.451209 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:41:14.029364 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:14.029329 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fzqtb/must-gather-8rctm" event={"ID":"4b46555c-cea3-40c3-992f-8b966efbab8b","Type":"ContainerStarted","Data":"9362be96ef2cafd049be21580637a254ff4dde4573a95c36aaa8f6d33cc02c3e"} Jul 28 17:41:20.067952 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:20.067860 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fzqtb/must-gather-8rctm" event={"ID":"4b46555c-cea3-40c3-992f-8b966efbab8b","Type":"ContainerStarted","Data":"af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb"} Jul 28 17:41:20.067952 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:20.067913 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fzqtb/must-gather-8rctm" event={"ID":"4b46555c-cea3-40c3-992f-8b966efbab8b","Type":"ContainerStarted","Data":"f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e"} Jul 28 17:41:20.085072 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:20.085001 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fzqtb/must-gather-8rctm" podStartSLOduration=1.81633989 podStartE2EDuration="8.084980233s" totalImagesPullingTime="6.268640343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:20.08438377 +0000 UTC m=+1962.843244685" watchObservedRunningTime="2026-07-28 17:41:20.084980233 +0000 UTC m=+1962.843841147" Jul 28 17:41:30.111456 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:30.111370 2570 generic.go:356] "Generic (PLEG): container finished" podID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerID="f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e" exitCode=0 Jul 28 17:41:30.111860 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:30.111445 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fzqtb/must-gather-8rctm" event={"ID":"4b46555c-cea3-40c3-992f-8b966efbab8b","Type":"ContainerDied","Data":"f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e"} Jul 28 17:41:30.111860 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:30.111777 2570 scope.go:117] "RemoveContainer" containerID="f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e" Jul 28 17:41:30.349167 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:30.349138 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fzqtb_must-gather-8rctm_4b46555c-cea3-40c3-992f-8b966efbab8b/gather/0.log" Jul 28 17:41:33.360198 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:33.360173 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-62dsj_5feed6d1-3fca-42ac-991a-4803dbe46b52/global-pull-secret-syncer/0.log" Jul 28 17:41:33.574226 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:33.574197 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-kcdsz_2a9b8086-4250-4feb-baee-b561b9015044/konnectivity-agent/0.log" Jul 28 17:41:33.663300 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:33.663220 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-215.ec2.internal_4746d2d48c5a0af4bf6819c4fa3b9ae9/haproxy/0.log" Jul 28 17:41:35.693470 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:35.693435 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-fzqtb/must-gather-8rctm"] Jul 28 17:41:35.694377 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:35.694313 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-fzqtb/must-gather-8rctm" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerName="copy" containerID="cri-o://af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb" gracePeriod=2 Jul 28 17:41:35.696708 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:41:35.696671 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-8rctm\" is forbidden: User \"system:node:ip-10-0-136-215.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-fzqtb\": no relationship found between node 'ip-10-0-136-215.ec2.internal' and this object" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:35.699318 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:35.699290 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-fzqtb/must-gather-8rctm"] Jul 28 17:41:35.929501 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:35.929475 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fzqtb_must-gather-8rctm_4b46555c-cea3-40c3-992f-8b966efbab8b/copy/0.log" Jul 28 17:41:35.931113 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:35.931091 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:36.030459 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.030373 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b46555c-cea3-40c3-992f-8b966efbab8b-must-gather-output\") pod \"4b46555c-cea3-40c3-992f-8b966efbab8b\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " Jul 28 17:41:36.030459 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.030410 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtqkx\" (UniqueName: \"kubernetes.io/projected/4b46555c-cea3-40c3-992f-8b966efbab8b-kube-api-access-mtqkx\") pod \"4b46555c-cea3-40c3-992f-8b966efbab8b\" (UID: \"4b46555c-cea3-40c3-992f-8b966efbab8b\") " Jul 28 17:41:36.030752 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.030728 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b46555c-cea3-40c3-992f-8b966efbab8b-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "4b46555c-cea3-40c3-992f-8b966efbab8b" (UID: "4b46555c-cea3-40c3-992f-8b966efbab8b"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:41:36.032590 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.032563 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b46555c-cea3-40c3-992f-8b966efbab8b-kube-api-access-mtqkx" (OuterVolumeSpecName: "kube-api-access-mtqkx") pod "4b46555c-cea3-40c3-992f-8b966efbab8b" (UID: "4b46555c-cea3-40c3-992f-8b966efbab8b"). InnerVolumeSpecName "kube-api-access-mtqkx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:41:36.131295 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.131258 2570 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4b46555c-cea3-40c3-992f-8b966efbab8b-must-gather-output\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:41:36.131295 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.131288 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtqkx\" (UniqueName: \"kubernetes.io/projected/4b46555c-cea3-40c3-992f-8b966efbab8b-kube-api-access-mtqkx\") on node \"ip-10-0-136-215.ec2.internal\" DevicePath \"\"" Jul 28 17:41:36.136907 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.136885 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-fzqtb_must-gather-8rctm_4b46555c-cea3-40c3-992f-8b966efbab8b/copy/0.log" Jul 28 17:41:36.137284 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.137263 2570 generic.go:356] "Generic (PLEG): container finished" podID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerID="af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb" exitCode=143 Jul 28 17:41:36.137330 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.137313 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fzqtb/must-gather-8rctm" Jul 28 17:41:36.137383 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.137369 2570 scope.go:117] "RemoveContainer" containerID="af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb" Jul 28 17:41:36.146670 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.146647 2570 scope.go:117] "RemoveContainer" containerID="f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e" Jul 28 17:41:36.158769 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.158747 2570 scope.go:117] "RemoveContainer" containerID="af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb" Jul 28 17:41:36.159076 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:41:36.159056 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb\": container with ID starting with af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb not found: ID does not exist" containerID="af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb" Jul 28 17:41:36.159129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.159087 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb"} err="failed to get container status \"af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb\": rpc error: code = NotFound desc = could not find container \"af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb\": container with ID starting with af77efbdea1381a95eaadde790fad2b2152b8666b067f010935343f8347da6cb not found: ID does not exist" Jul 28 17:41:36.159129 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.159107 2570 scope.go:117] "RemoveContainer" containerID="f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e" Jul 28 17:41:36.159371 ip-10-0-136-215 kubenswrapper[2570]: E0728 17:41:36.159350 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e\": container with ID starting with f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e not found: ID does not exist" containerID="f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e" Jul 28 17:41:36.159432 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.159377 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e"} err="failed to get container status \"f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e\": rpc error: code = NotFound desc = could not find container \"f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e\": container with ID starting with f44cd0a89d759c2df4722f929d384517815df6876908476a327c86c43d39367e not found: ID does not exist" Jul 28 17:41:36.996737 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:36.996710 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7673ac2-4d8f-431b-873b-49a606675b24/alertmanager/0.log" Jul 28 17:41:37.036174 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.036149 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7673ac2-4d8f-431b-873b-49a606675b24/config-reloader/0.log" Jul 28 17:41:37.076709 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.076683 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7673ac2-4d8f-431b-873b-49a606675b24/kube-rbac-proxy-web/0.log" Jul 28 17:41:37.110285 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.110262 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7673ac2-4d8f-431b-873b-49a606675b24/kube-rbac-proxy/0.log" Jul 28 17:41:37.142776 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.142745 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7673ac2-4d8f-431b-873b-49a606675b24/kube-rbac-proxy-metric/0.log" Jul 28 17:41:37.181071 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.181040 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7673ac2-4d8f-431b-873b-49a606675b24/prom-label-proxy/0.log" Jul 28 17:41:37.212736 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.212713 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7673ac2-4d8f-431b-873b-49a606675b24/init-config-reloader/0.log" Jul 28 17:41:37.256008 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.255935 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-thnp6_cf74e9e9-bb2e-4aa6-bdad-bd1525a43b86/cluster-monitoring-operator/0.log" Jul 28 17:41:37.381173 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.381149 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7545c9ffff-7km2p_639095c0-04a3-4121-99a2-3acf77a4c993/metrics-server/0.log" Jul 28 17:41:37.415346 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.415320 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-k9qnv_471dd41e-aae6-424e-9c09-7f24e42f012c/monitoring-plugin/0.log" Jul 28 17:41:37.454937 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.454911 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mdxh_0b4c68a3-7f4e-4048-a86a-e6163167754e/node-exporter/0.log" Jul 28 17:41:37.483560 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.483537 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mdxh_0b4c68a3-7f4e-4048-a86a-e6163167754e/kube-rbac-proxy/0.log" Jul 28 17:41:37.510062 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.509991 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5mdxh_0b4c68a3-7f4e-4048-a86a-e6163167754e/init-textfile/0.log" Jul 28 17:41:37.723932 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:37.723900 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" path="/var/lib/kubelet/pods/4b46555c-cea3-40c3-992f-8b966efbab8b/volumes" Jul 28 17:41:38.088928 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.088901 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-pkgcg_5003b68a-bbf3-4382-b934-7c5371c76bcc/prometheus-operator/0.log" Jul 28 17:41:38.118264 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.118229 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-pkgcg_5003b68a-bbf3-4382-b934-7c5371c76bcc/kube-rbac-proxy/0.log" Jul 28 17:41:38.153405 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.153332 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-n4qrw_c435ba02-2d0b-4e5b-9d28-91ebae5043b3/prometheus-operator-admission-webhook/0.log" Jul 28 17:41:38.278996 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.278971 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97b7b6f55-7287d_00e1ef9a-175d-40d0-9cf0-4a2661c4c606/thanos-query/0.log" Jul 28 17:41:38.305206 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.305179 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97b7b6f55-7287d_00e1ef9a-175d-40d0-9cf0-4a2661c4c606/kube-rbac-proxy-web/0.log" Jul 28 17:41:38.337377 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.337348 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97b7b6f55-7287d_00e1ef9a-175d-40d0-9cf0-4a2661c4c606/kube-rbac-proxy/0.log" Jul 28 17:41:38.368472 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.368447 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97b7b6f55-7287d_00e1ef9a-175d-40d0-9cf0-4a2661c4c606/prom-label-proxy/0.log" Jul 28 17:41:38.396551 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.396521 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97b7b6f55-7287d_00e1ef9a-175d-40d0-9cf0-4a2661c4c606/kube-rbac-proxy-rules/0.log" Jul 28 17:41:38.423160 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:38.423083 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-97b7b6f55-7287d_00e1ef9a-175d-40d0-9cf0-4a2661c4c606/kube-rbac-proxy-metrics/0.log" Jul 28 17:41:39.707815 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:39.707789 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-9522p_3b79145a-1a31-4970-82af-1212de987dc9/networking-console-plugin/0.log" Jul 28 17:41:41.155888 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.155851 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd"] Jul 28 17:41:41.156266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.156189 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerName="copy" Jul 28 17:41:41.156266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.156200 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerName="copy" Jul 28 17:41:41.156266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.156217 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerName="gather" Jul 28 17:41:41.156266 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.156222 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerName="gather" Jul 28 17:41:41.156400 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.156279 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerName="copy" Jul 28 17:41:41.156400 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.156287 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b46555c-cea3-40c3-992f-8b966efbab8b" containerName="gather" Jul 28 17:41:41.158243 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.158222 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.160838 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.160807 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gvbm4\"/\"kube-root-ca.crt\"" Jul 28 17:41:41.161994 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.161965 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gvbm4\"/\"openshift-service-ca.crt\"" Jul 28 17:41:41.162097 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.161963 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gvbm4\"/\"default-dockercfg-ptdtn\"" Jul 28 17:41:41.166948 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.166903 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22fcs\" (UniqueName: \"kubernetes.io/projected/b68c899b-ec56-415d-b3db-65a425aae26d-kube-api-access-22fcs\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.167251 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.166960 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-lib-modules\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.167380 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.167331 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-sys\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.167380 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.167371 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-podres\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.167502 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.167388 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-proc\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.168598 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.168580 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd"] Jul 28 17:41:41.193214 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.193187 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-qscf7_1d65527e-a4a8-465f-81c0-2bd8b441d91b/volume-data-source-validator/0.log" Jul 28 17:41:41.268799 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.268759 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22fcs\" (UniqueName: \"kubernetes.io/projected/b68c899b-ec56-415d-b3db-65a425aae26d-kube-api-access-22fcs\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269019 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.268814 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-lib-modules\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269019 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.268867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-sys\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269019 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.268897 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-podres\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269019 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.268914 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-proc\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269019 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.268991 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-sys\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269220 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.269002 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-proc\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269220 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.269051 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-lib-modules\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.269220 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.269055 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b68c899b-ec56-415d-b3db-65a425aae26d-podres\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.279663 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.279626 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22fcs\" (UniqueName: \"kubernetes.io/projected/b68c899b-ec56-415d-b3db-65a425aae26d-kube-api-access-22fcs\") pod \"perf-node-gather-daemonset-sfwzd\" (UID: \"b68c899b-ec56-415d-b3db-65a425aae26d\") " pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.468713 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.468622 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:41.596268 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:41.596244 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd"] Jul 28 17:41:41.598414 ip-10-0-136-215 kubenswrapper[2570]: W0728 17:41:41.598382 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb68c899b_ec56_415d_b3db_65a425aae26d.slice/crio-62f530faa43c0841fbe40ce7fadfe0557a04b072efa3dae07230ae79f3ce0966 WatchSource:0}: Error finding container 62f530faa43c0841fbe40ce7fadfe0557a04b072efa3dae07230ae79f3ce0966: Status 404 returned error can't find the container with id 62f530faa43c0841fbe40ce7fadfe0557a04b072efa3dae07230ae79f3ce0966 Jul 28 17:41:42.107369 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.107312 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mjf49_36520022-e419-412e-bf8b-d24c5da77c7d/dns/0.log" Jul 28 17:41:42.133795 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.133765 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mjf49_36520022-e419-412e-bf8b-d24c5da77c7d/kube-rbac-proxy/0.log" Jul 28 17:41:42.168187 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.168153 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" event={"ID":"b68c899b-ec56-415d-b3db-65a425aae26d","Type":"ContainerStarted","Data":"e1a1b770320d04d249e27d17d9c46ba3a8a848a3bfec6b601567e1a65ac91a55"} Jul 28 17:41:42.168560 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.168209 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" event={"ID":"b68c899b-ec56-415d-b3db-65a425aae26d","Type":"ContainerStarted","Data":"62f530faa43c0841fbe40ce7fadfe0557a04b072efa3dae07230ae79f3ce0966"} Jul 28 17:41:42.168560 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.168309 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:42.185466 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.185415 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" podStartSLOduration=1.185400338 podStartE2EDuration="1.185400338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:41:42.183718002 +0000 UTC m=+1984.942578949" watchObservedRunningTime="2026-07-28 17:41:42.185400338 +0000 UTC m=+1984.944261252" Jul 28 17:41:42.322458 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.322429 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vdrdq_48646f83-5284-4e24-8ea7-da85a74922aa/dns-node-resolver/0.log" Jul 28 17:41:42.827238 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:42.827206 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vbj6k_ba17a781-9e7c-4ec2-ab60-9a2d4132c464/node-ca/0.log" Jul 28 17:41:43.975084 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:43.975055 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r2rw6_c00ba8b0-d3ea-4a1c-8967-6d48517b3568/serve-healthcheck-canary/0.log" Jul 28 17:41:44.395299 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:44.395267 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-z6qvn_a5530c61-043a-404e-afad-43651d677628/insights-operator/0.log" Jul 28 17:41:44.395610 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:44.395592 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-z6qvn_a5530c61-043a-404e-afad-43651d677628/insights-operator/1.log" Jul 28 17:41:44.492362 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:44.492330 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pr6k7_2321871d-b9e6-48c0-be39-201edbf8906e/kube-rbac-proxy/0.log" Jul 28 17:41:44.518248 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:44.518226 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pr6k7_2321871d-b9e6-48c0-be39-201edbf8906e/exporter/0.log" Jul 28 17:41:44.544682 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:44.544656 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pr6k7_2321871d-b9e6-48c0-be39-201edbf8906e/extractor/0.log" Jul 28 17:41:48.181671 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:48.181639 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gvbm4/perf-node-gather-daemonset-sfwzd" Jul 28 17:41:49.279661 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:49.279635 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-nvvw8_1180225f-124f-4b95-877d-de5695356f61/migrator/0.log" Jul 28 17:41:49.305124 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:49.305092 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-nvvw8_1180225f-124f-4b95-877d-de5695356f61/graceful-termination/0.log" Jul 28 17:41:49.665443 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:49.665407 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh_b1525dd4-496e-4b33-964e-db81ff558ed9/kube-storage-version-migrator-operator/1.log" Jul 28 17:41:49.666311 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:49.666295 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-7jvgh_b1525dd4-496e-4b33-964e-db81ff558ed9/kube-storage-version-migrator-operator/0.log" Jul 28 17:41:50.904215 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:50.904186 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9hbp_90d0c9d9-154c-4867-acd4-3c4a14069201/kube-multus-additional-cni-plugins/0.log" Jul 28 17:41:50.932443 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:50.932420 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9hbp_90d0c9d9-154c-4867-acd4-3c4a14069201/egress-router-binary-copy/0.log" Jul 28 17:41:50.956155 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:50.956133 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9hbp_90d0c9d9-154c-4867-acd4-3c4a14069201/cni-plugins/0.log" Jul 28 17:41:50.985584 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:50.985555 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9hbp_90d0c9d9-154c-4867-acd4-3c4a14069201/bond-cni-plugin/0.log" Jul 28 17:41:51.010923 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:51.010900 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9hbp_90d0c9d9-154c-4867-acd4-3c4a14069201/routeoverride-cni/0.log" Jul 28 17:41:51.036180 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:51.036157 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9hbp_90d0c9d9-154c-4867-acd4-3c4a14069201/whereabouts-cni-bincopy/0.log" Jul 28 17:41:51.060912 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:51.060886 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-c9hbp_90d0c9d9-154c-4867-acd4-3c4a14069201/whereabouts-cni/0.log" Jul 28 17:41:51.278414 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:51.278337 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-t9m7d_e569df22-dae2-4b3d-b50a-6fda9da8140b/kube-multus/0.log" Jul 28 17:41:51.407927 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:51.407903 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-plwl2_b00417f8-bed7-4fcb-b3ec-f912fc8dfb91/network-metrics-daemon/0.log" Jul 28 17:41:51.430494 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:51.430466 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-plwl2_b00417f8-bed7-4fcb-b3ec-f912fc8dfb91/kube-rbac-proxy/0.log" Jul 28 17:41:52.784670 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.784591 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/ovn-controller/0.log" Jul 28 17:41:52.805974 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.805940 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/ovn-acl-logging/0.log" Jul 28 17:41:52.837407 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.837381 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/kube-rbac-proxy-node/0.log" Jul 28 17:41:52.860637 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.860607 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 17:41:52.880879 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.880852 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/northd/0.log" Jul 28 17:41:52.905216 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.905189 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/nbdb/0.log" Jul 28 17:41:52.929725 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.929696 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/sbdb/0.log" Jul 28 17:41:52.976731 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:52.976700 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r9ndn_987609de-891a-47cb-b2da-262a39bda2fa/ovnkube-controller/0.log" Jul 28 17:41:53.960455 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:53.960429 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-q247t_358e9a0a-b26b-4b6a-b53c-2d83fea72f48/check-endpoints/0.log" Jul 28 17:41:54.044042 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:54.044015 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-qtjtr_93cfcf12-af34-48a1-81dd-4957742144ff/network-check-target-container/0.log" Jul 28 17:41:55.036490 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:55.036442 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-tp7cx_cc70882a-a450-4afd-9a42-8eaa3bcc61d6/iptables-alerter/0.log" Jul 28 17:41:55.719269 ip-10-0-136-215 kubenswrapper[2570]: I0728 17:41:55.719240 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-d92lh_8e93e778-0d32-4bf2-8be0-7999c020ae7e/tuned/0.log"