Jul 17 13:17:16.598096 ip-10-0-142-189 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 17 13:17:16.598111 ip-10-0-142-189 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 17 13:17:16.598120 ip-10-0-142-189 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 17 13:17:16.598449 ip-10-0-142-189 systemd[1]: Failed to start Kubernetes Kubelet. Jul 17 13:17:26.742357 ip-10-0-142-189 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 17 13:17:26.742378 ip-10-0-142-189 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 906859d1582c456ca9cc32600935e5cb -- Jul 17 13:19:15.682712 ip-10-0-142-189 systemd[1]: Starting Kubernetes Kubelet... Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.078162 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082772 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082785 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082790 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082793 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082797 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082800 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:19:16.167685 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082803 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:19:16.145857 ip-10-0-142-189 systemd[1]: Started Kubernetes Kubelet. Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082806 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082808 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082811 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082814 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082816 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082819 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082827 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082830 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082833 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082836 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082838 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082841 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082844 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082847 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082849 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082852 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082855 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082858 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:19:16.169117 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082861 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082866 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082869 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082872 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082875 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082878 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082881 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082883 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082886 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082888 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082891 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082893 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082896 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082898 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082901 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082903 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082906 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082908 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082911 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082914 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:19:16.170176 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082916 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082919 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082921 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082924 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082926 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082929 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082931 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082935 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082937 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082940 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082945 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082948 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082950 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082953 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082956 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082958 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082961 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082963 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082966 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082969 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:19:16.171504 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082971 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082975 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082977 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082980 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082982 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082986 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082990 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082993 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082996 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.082999 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083002 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083005 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083007 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083010 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083013 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083015 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083018 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083020 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083023 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:19:16.172577 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083025 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083028 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083031 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083033 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083036 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083038 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083041 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083044 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083046 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083049 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083051 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083054 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083056 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083058 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.083061 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084399 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084407 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084411 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084414 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084418 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:19:16.173800 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084421 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084424 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084427 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084430 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084432 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084435 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084438 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084441 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084444 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084447 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084449 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084452 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084455 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084471 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084474 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084477 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084480 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084482 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084485 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084487 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:19:16.178972 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084490 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084494 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084498 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084501 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084504 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084506 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084511 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084527 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084530 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084533 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084537 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084540 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084542 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084546 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084549 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084552 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084554 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084557 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084560 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:19:16.179845 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084563 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084566 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084569 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084572 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084575 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084577 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084580 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084582 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084585 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084588 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084590 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084593 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084595 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084598 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084601 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084603 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084606 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084610 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084614 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:19:16.180721 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084617 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084619 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084624 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084627 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084630 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084633 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084635 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084638 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084641 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084644 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084647 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084649 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084652 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084655 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084657 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084660 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084663 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084666 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084668 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084671 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:19:16.182672 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084673 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084676 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084679 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084682 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084684 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084687 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084690 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084692 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084695 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084697 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084700 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084703 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084705 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084708 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084710 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.084713 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085071 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085076 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085078 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:19:16.183581 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085081 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085084 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085087 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085090 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085093 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085096 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085098 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085101 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085104 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085106 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085109 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085112 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085114 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085117 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085120 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085123 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085125 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085128 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085131 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085134 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:19:16.184623 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085136 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085139 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085142 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085144 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085147 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085152 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085154 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085157 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085160 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085163 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085165 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085168 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085170 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085173 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085179 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085182 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085184 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085187 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085189 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085192 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:19:16.186059 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085195 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085197 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085200 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085202 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085205 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085208 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085212 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085216 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085219 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085222 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085225 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085227 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085230 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085232 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085235 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085237 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085240 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085244 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085246 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:19:16.186872 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085249 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085251 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085254 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085256 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085259 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085261 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085264 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085267 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085269 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085272 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085275 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085278 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085281 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085285 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085288 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085292 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085295 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085297 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085301 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:19:16.188321 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085304 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085306 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085309 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085311 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085314 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085317 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085321 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085324 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085327 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085330 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085332 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085337 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085339 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085342 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085345 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085347 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085350 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.085352 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085419 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085426 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085431 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 13:19:16.189154 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085435 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085454 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085470 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085474 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085478 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085481 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085484 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085487 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085491 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085494 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085499 2560 flags.go:64] FLAG: --cgroup-root="" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085502 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085505 2560 flags.go:64] FLAG: --client-ca-file="" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085508 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085511 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085520 2560 flags.go:64] FLAG: --cluster-domain="" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085523 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085526 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085529 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085533 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085536 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085539 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085543 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 13:19:16.190426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085547 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085550 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085553 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085555 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085558 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085561 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085565 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085568 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085571 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085574 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085577 2560 flags.go:64] FLAG: --enable-server="true" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085582 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085589 2560 flags.go:64] FLAG: --event-burst="100" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085592 2560 flags.go:64] FLAG: --event-qps="50" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085595 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085598 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085601 2560 flags.go:64] FLAG: --eviction-hard="" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085605 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085608 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085612 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085615 2560 flags.go:64] FLAG: --eviction-soft="" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085618 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085621 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085624 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085626 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 13:19:16.191647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085629 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085632 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085635 2560 flags.go:64] FLAG: --feature-gates="" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085639 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085642 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085645 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085649 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085653 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085655 2560 flags.go:64] FLAG: --help="false" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085658 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-142-189.ec2.internal" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085661 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085664 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085667 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085670 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085674 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085677 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085679 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085682 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085685 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085688 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085691 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085694 2560 flags.go:64] FLAG: --kube-reserved="" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085697 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085700 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 13:19:16.194083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085703 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085705 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085709 2560 flags.go:64] FLAG: --lock-file="" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085712 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085715 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085717 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085722 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085725 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085728 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085731 2560 flags.go:64] FLAG: --logging-format="text" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085734 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085737 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085740 2560 flags.go:64] FLAG: --manifest-url="" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085743 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085748 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085751 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085755 2560 flags.go:64] FLAG: --max-pods="110" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085758 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085761 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085764 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085767 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085770 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085773 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085776 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085782 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 13:19:16.195315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085785 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085788 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085791 2560 flags.go:64] FLAG: --pod-cidr="" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085794 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085800 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085802 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085805 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085808 2560 flags.go:64] FLAG: --port="10250" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085811 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085815 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0979882d836703c3a" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085818 2560 flags.go:64] FLAG: --qos-reserved="" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085822 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085825 2560 flags.go:64] FLAG: --register-node="true" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085828 2560 flags.go:64] FLAG: --register-with-taints="" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085831 2560 flags.go:64] FLAG: --registry-burst="10" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085834 2560 flags.go:64] FLAG: --registry-qps="5" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085837 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085839 2560 flags.go:64] FLAG: --reserved-memory="" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085842 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085845 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085848 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085851 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085855 2560 flags.go:64] FLAG: --runonce="false" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085858 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085861 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 13:19:16.197008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085864 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085867 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085870 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085873 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085875 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085878 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085881 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085884 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085887 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085889 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085892 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085896 2560 flags.go:64] FLAG: --system-cgroups="" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085899 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085904 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085906 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085909 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085914 2560 flags.go:64] FLAG: --tls-min-version="" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085917 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085920 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085923 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085926 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085929 2560 flags.go:64] FLAG: --v="2" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085932 2560 flags.go:64] FLAG: --version="false" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085936 2560 flags.go:64] FLAG: --vmodule="" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085939 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 13:19:16.198227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.085942 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086058 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086062 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086065 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086070 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086072 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086075 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086078 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086081 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086084 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086086 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086089 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086092 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086094 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086097 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086099 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086102 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086104 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086107 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086110 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:19:16.199859 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086112 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086115 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086117 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086121 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086124 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086126 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086129 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086131 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086134 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086136 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086139 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086142 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086144 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086147 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086150 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086152 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086156 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086159 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086162 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086164 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:19:16.200676 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086167 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086170 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086172 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086175 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086177 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086180 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086183 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086186 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086188 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086191 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086195 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086198 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086202 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086205 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086209 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086213 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086215 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086218 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086220 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:19:16.202757 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086223 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086225 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086228 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086230 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086233 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086236 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086238 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086240 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086243 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086247 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086249 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086252 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086255 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086257 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086260 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086262 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086265 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086267 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086270 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086272 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:19:16.203559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086275 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086278 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086280 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086283 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086285 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086289 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086293 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086297 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086302 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086305 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086308 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086311 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086314 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086316 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086319 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086322 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086324 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086327 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086329 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:19:16.206009 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086332 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.086334 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.086341 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.093106 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.093121 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.093139 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.093146 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093200 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093205 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093209 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093212 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093216 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093219 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093221 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093224 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:19:16.206839 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093228 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093232 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093237 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093240 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093242 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093246 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093248 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093251 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093254 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093257 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093259 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093264 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093267 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093270 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093273 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093276 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093279 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093282 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093285 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:19:16.208508 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093288 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093291 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093295 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093298 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093301 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093303 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093306 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093309 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093312 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093314 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093317 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093320 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093326 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093329 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093331 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093334 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093337 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093339 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093342 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:19:16.209279 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093345 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093348 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093350 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093353 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093356 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093358 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093361 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093363 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093366 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093368 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093371 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093373 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093376 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093378 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093381 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093384 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093387 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093389 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093393 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093395 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:19:16.210085 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093400 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093402 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093405 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093408 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093410 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093414 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093416 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093419 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093421 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093424 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093427 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093429 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093431 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093434 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093437 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093440 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093442 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093445 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093449 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:19:16.210888 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093452 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093456 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093471 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093474 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093477 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093480 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093483 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093486 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093489 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093491 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093495 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093497 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093500 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093504 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.093509 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 13:19:16.213784 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093635 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093641 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093645 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093649 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093654 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093657 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093660 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093662 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093665 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093668 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093671 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093673 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093676 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093679 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093682 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093684 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093687 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093690 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093692 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093695 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 13:19:16.214991 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093698 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093701 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093705 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093708 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093710 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093713 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093715 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093718 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093721 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093724 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093728 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093731 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093733 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093735 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093738 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093741 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093745 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093747 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093750 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093752 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 13:19:16.217038 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093755 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093758 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093760 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093763 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093767 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093770 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093774 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093777 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093780 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093783 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093785 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093788 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093791 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093794 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093797 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093800 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093802 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093805 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093808 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 13:19:16.217865 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093810 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093813 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093816 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093819 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093822 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093825 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093827 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093830 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093832 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093836 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093839 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093841 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093844 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093846 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093849 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093852 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093854 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093857 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093859 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093862 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 13:19:16.327424 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093865 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093868 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093870 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093873 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093875 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093878 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093880 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093882 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093885 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093887 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093890 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093893 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093895 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093898 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093900 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093904 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093907 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093909 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093912 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 13:19:16.328216 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:16.093915 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.093919 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.094010 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.096074 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.097339 2560 server.go:1019] "Starting client certificate rotation" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.097440 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.097482 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.101148 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.104233 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.118501 2560 log.go:25] "Validated CRI v1 runtime API" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.120409 2560 log.go:25] "Validated CRI v1 image API" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.121276 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 13:19:16.330242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.123297 2560 fs.go:135] Filesystem UUIDs: map[6852520c-f397-47ec-9660-7fcc51505544:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a9c5c7bc-ad76-4d39-9705-3b7734019d74:/dev/nvme0n1p4] Jul 17 13:19:16.330656 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.123312 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 13:19:16.330656 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.125474 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.128182 2560 manager.go:216] Machine: {Timestamp:2026-07-17 13:19:16.126844197 +0000 UTC m=+0.327406744 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101609 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2abc03116914ff93427aaff3f79beb SystemUUID:ec2abc03-1169-14ff-9342-7aaff3f79beb BootID:906859d1-582c-456c-a9cc-32600935e5cb Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b5:3f:be:1a:43 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b5:3f:be:1a:43 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:06:1a:6f:a6:28 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.128276 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.128344 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.131324 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.131350 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-189.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.132148 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.132156 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.132164 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.132782 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.133900 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.136623 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.136728 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.136754 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.136763 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.136777 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.136786 2560 kubelet.go:398] "Adding apiserver pod source" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.136794 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 13:19:16.332679 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.137121 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.137139 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.139061 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.140558 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.140585 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141099 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141112 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141118 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141123 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141132 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141138 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141227 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141234 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141241 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141247 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141255 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141263 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141880 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.141891 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.145128 2560 server.go:1268] "Started kubelet" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.145193 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.145223 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.145298 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.149491 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-189.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.150563 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.150611 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.150839 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.152419 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8gsjd" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.154653 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.154689 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.154582 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-189.ec2.internal.18c3164368c02ca1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-189.ec2.internal,UID:ip-10-0-142-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-189.ec2.internal,},FirstTimestamp:2026-07-17 13:19:16.144893089 +0000 UTC m=+0.345455636,LastTimestamp:2026-07-17 13:19:16.144893089 +0000 UTC m=+0.345455636,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-189.ec2.internal,}" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.156656 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.156675 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.156737 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.157325 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.157346 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.157326 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.157494 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.157507 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158061 2560 factory.go:55] Registering systemd factory Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158092 2560 factory.go:223] Registration of the systemd container factory successfully Jul 17 13:19:16.335108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158254 2560 factory.go:153] Registering CRI-O factory Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158268 2560 factory.go:223] Registration of the crio container factory successfully Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158317 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158339 2560 factory.go:103] Registering Raw factory Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158353 2560 manager.go:1196] Started watching for new ooms in manager Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.158767 2560 manager.go:349] Starting recovery of all containers Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.159947 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.160028 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-189.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.160120 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8gsjd" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.168597 2560 manager.go:354] Recovery completed Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.175342 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.178127 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.178156 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.178166 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.178717 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.178733 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.178739 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.178749 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.179474 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.179501 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.179515 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.179820 2560 policy_none.go:49] "None policy: Start" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.179835 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.179850 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.179826 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-189.ec2.internal.18c316436abb827b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-189.ec2.internal,UID:ip-10-0-142-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-189.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-189.ec2.internal,},FirstTimestamp:2026-07-17 13:19:16.178141819 +0000 UTC m=+0.378704366,LastTimestamp:2026-07-17 13:19:16.178141819 +0000 UTC m=+0.378704366,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-189.ec2.internal,}" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:16.181057 2560 policy_none.go:47] "Start" Jul 17 13:19:16.336311 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.257050 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:16.448019 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.357315 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:16.448019 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.364993 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-189.ec2.internal\" not found" node="ip-10-0-142-189.ec2.internal" Jul 17 13:19:16.459119 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.458203 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:16.572144 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.558571 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:16.689091 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.659202 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:16.795741 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:16.759546 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:18.982335 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:18.982185 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.001582 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.001562 2560 manager.go:337] "Starting Device Plugin manager" Jul 17 13:19:19.001688 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.001603 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 13:19:19.001688 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.001616 2560 server.go:87] "Starting device plugin registration server" Jul 17 13:19:19.001935 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.001898 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 13:19:19.002040 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.001933 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 13:19:19.002577 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.002175 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 13:19:19.002577 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.002270 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 13:19:19.002577 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.002279 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 13:19:19.002764 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.002707 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 13:19:19.002764 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.002759 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.093880 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.093851 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 13:19:19.102741 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.095006 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 13:19:19.102741 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.095042 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 13:19:19.102741 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.095063 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 13:19:19.102741 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.095096 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 17 13:19:19.102741 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.097034 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:19:19.102956 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.102944 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:19:19.103680 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.103662 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:19:19.103760 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.103691 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:19:19.103760 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.103712 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:19:19.103760 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.103733 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.110751 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.110736 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.110819 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.110757 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-189.ec2.internal\": node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.120848 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.120827 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.195499 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.195463 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal"] Jul 17 13:19:19.195645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.195533 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:19:19.200107 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.200090 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:19:19.200186 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.200120 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:19:19.200186 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.200130 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:19:19.201448 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.201436 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:19:19.201563 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.201548 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.201598 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.201581 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:19:19.202130 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.202109 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:19:19.202130 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.202131 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:19:19.202130 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.202143 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:19:19.202323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.202196 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:19:19.202323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.202216 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:19:19.202323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.202230 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:19:19.204085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.204071 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.204145 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.204096 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 13:19:19.204779 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.204765 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientMemory" Jul 17 13:19:19.204838 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.204791 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 13:19:19.204838 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.204807 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeHasSufficientPID" Jul 17 13:19:19.219016 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.219001 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 13:19:19.219169 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.219150 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 13:19:19.219235 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.219167 2560 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a68fa96c039c04c568dbec7439b665bc-be49f50109ba4832.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-142-189.ec2.internal.18c316436abbcc4c\": read tcp 10.0.142.189:45066->18.208.254.67:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-142-189.ec2.internal.18c316436abbcc4c default 6455 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-189.ec2.internal,UID:ip-10-0-142-189.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-189.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-189.ec2.internal,},FirstTimestamp:2026-07-17 13:19:16 +0000 UTC,LastTimestamp:2026-07-17 13:19:19.200124674 +0000 UTC m=+1.278238329,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-189.ec2.internal,}" Jul 17 13:19:19.221156 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.221142 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.230499 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.230480 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-189.ec2.internal\" not found" node="ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.234877 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.234863 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-189.ec2.internal\" not found" node="ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.282615 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.282598 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:19:19.283627 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.283612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe62c43e5ee9d2052903089d10f2a507-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal\" (UID: \"fe62c43e5ee9d2052903089d10f2a507\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.283678 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.283634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/068abd2681734d05200c5d2715b73db9-config\") pod \"kube-apiserver-proxy-ip-10-0-142-189.ec2.internal\" (UID: \"068abd2681734d05200c5d2715b73db9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.283678 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.283651 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe62c43e5ee9d2052903089d10f2a507-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal\" (UID: \"fe62c43e5ee9d2052903089d10f2a507\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.288353 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.288331 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 13:14:18 +0000 UTC" deadline="2028-01-31 21:30:23.369339543 +0000 UTC" Jul 17 13:19:19.288414 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.288353 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13520h11m4.080989099s" Jul 17 13:19:19.321827 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.321806 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.384136 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.384112 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/068abd2681734d05200c5d2715b73db9-config\") pod \"kube-apiserver-proxy-ip-10-0-142-189.ec2.internal\" (UID: \"068abd2681734d05200c5d2715b73db9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.384136 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.384143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe62c43e5ee9d2052903089d10f2a507-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal\" (UID: \"fe62c43e5ee9d2052903089d10f2a507\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.384251 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.384159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe62c43e5ee9d2052903089d10f2a507-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal\" (UID: \"fe62c43e5ee9d2052903089d10f2a507\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.384251 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.384203 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe62c43e5ee9d2052903089d10f2a507-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal\" (UID: \"fe62c43e5ee9d2052903089d10f2a507\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.384251 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.384218 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/068abd2681734d05200c5d2715b73db9-config\") pod \"kube-apiserver-proxy-ip-10-0-142-189.ec2.internal\" (UID: \"068abd2681734d05200c5d2715b73db9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.384251 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.384231 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe62c43e5ee9d2052903089d10f2a507-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal\" (UID: \"fe62c43e5ee9d2052903089d10f2a507\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.422243 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.422225 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.522796 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.522728 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.531857 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.531839 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.538492 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.538472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.623456 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.623427 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.723955 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:19.723929 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-189.ec2.internal\" not found" Jul 17 13:19:19.748749 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.748725 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:19:19.749637 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.749622 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:19:19.779918 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.779851 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.789295 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.789278 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 13:19:19.792449 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.792436 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" Jul 17 13:19:19.800167 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:19.800148 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 13:19:20.003842 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:20.003795 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe62c43e5ee9d2052903089d10f2a507.slice/crio-d4240c24787da2d994042a481f33d4069a7bbe07dd40fa3ce8e3aaad17bc7bdd WatchSource:0}: Error finding container d4240c24787da2d994042a481f33d4069a7bbe07dd40fa3ce8e3aaad17bc7bdd: Status 404 returned error can't find the container with id d4240c24787da2d994042a481f33d4069a7bbe07dd40fa3ce8e3aaad17bc7bdd Jul 17 13:19:20.004189 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:20.004169 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068abd2681734d05200c5d2715b73db9.slice/crio-f130b289cf221398a7d6ac435ed55eb01551ae1b420e6414d1a716bde276ebbb WatchSource:0}: Error finding container f130b289cf221398a7d6ac435ed55eb01551ae1b420e6414d1a716bde276ebbb: Status 404 returned error can't find the container with id f130b289cf221398a7d6ac435ed55eb01551ae1b420e6414d1a716bde276ebbb Jul 17 13:19:20.007865 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.007849 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 13:19:20.098669 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.098587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" event={"ID":"fe62c43e5ee9d2052903089d10f2a507","Type":"ContainerStarted","Data":"d4240c24787da2d994042a481f33d4069a7bbe07dd40fa3ce8e3aaad17bc7bdd"} Jul 17 13:19:20.099606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.099580 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" event={"ID":"068abd2681734d05200c5d2715b73db9","Type":"ContainerStarted","Data":"f130b289cf221398a7d6ac435ed55eb01551ae1b420e6414d1a716bde276ebbb"} Jul 17 13:19:20.260769 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.260743 2560 apiserver.go:52] "Watching apiserver" Jul 17 13:19:20.270045 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.270018 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 13:19:20.272347 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.272325 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal","openshift-cluster-node-tuning-operator/tuned-m5nr4","openshift-multus/network-metrics-daemon-pfz9l","openshift-network-operator/iptables-alerter-wxgng","openshift-ovn-kubernetes/ovnkube-node-s478r","kube-system/konnectivity-agent-44d5g","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv","openshift-dns/node-resolver-52ncl","openshift-image-registry/node-ca-twxnh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal","openshift-multus/multus-additional-cni-plugins-lv575","openshift-multus/multus-t2xx7","openshift-network-diagnostics/network-check-target-8nhwn","kube-system/global-pull-secret-syncer-9b6c9"] Jul 17 13:19:20.274420 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.274404 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.276055 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.276039 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:20.276123 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.276102 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:20.276675 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.276658 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pwzws\"" Jul 17 13:19:20.276974 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.276958 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 13:19:20.277056 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.277022 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 13:19:20.277212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.277199 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.278739 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.278721 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.279803 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.279774 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 13:19:20.279803 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.279792 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cxxl5\"" Jul 17 13:19:20.279979 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.279798 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 13:19:20.279979 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.279836 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 13:19:20.279979 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.279871 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.280135 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.280017 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 13:19:20.281544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.281382 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 13:19:20.281544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.281383 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 13:19:20.281544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.281421 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 13:19:20.281544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.281435 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 13:19:20.281544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.281424 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 13:19:20.281812 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.281560 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.282499 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.282480 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 13:19:20.282608 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.282557 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 13:19:20.282608 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.282588 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 13:19:20.282608 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.282602 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.282764 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.282687 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cdnf8\"" Jul 17 13:19:20.282834 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.282796 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-86shz\"" Jul 17 13:19:20.283988 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.283973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.284112 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.284095 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 13:19:20.284327 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.284296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 13:19:20.284389 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.284374 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tf5dz\"" Jul 17 13:19:20.284458 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.284377 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 13:19:20.285082 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.285063 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 13:19:20.285369 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.285349 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8t67x\"" Jul 17 13:19:20.285459 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.285388 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 13:19:20.285809 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.285794 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.286587 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.286566 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 13:19:20.286681 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.286613 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 13:19:20.286681 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.286617 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 13:19:20.286681 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.286630 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l6w8x\"" Jul 17 13:19:20.287220 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287200 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.287857 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287838 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-kubernetes\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.288002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287861 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysctl-d\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.288002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287876 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-run\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.288002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-sys\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.288002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287937 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-slash\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.288002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287974 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-cni-netd\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.288002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.287997 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-sys-fs\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.288381 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.288022 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-var-lib-kubelet\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.288381 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.288056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-tuned\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.288381 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.288107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkc9x\" (UniqueName: \"kubernetes.io/projected/ff48e2f8-23d2-41e0-ad4e-ca761c116732-kube-api-access-pkc9x\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.288381 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.288142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-kubelet\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.288381 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.288171 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-var-lib-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.288381 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.288243 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.288381 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.288275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-ovn\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.289257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.289236 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:20.289354 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.289321 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:20.289794 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.289774 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 13:19:20.289899 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.289812 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-env-overrides\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.290315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.290295 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 13:19:20.290532 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.290513 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysconfig\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.290590 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.290552 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-lib-modules\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.290629 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.290586 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-tmp\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.290817 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.290801 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-j7kj6\"" Jul 17 13:19:20.290861 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.290843 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 13:19:20.291048 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291016 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff48e2f8-23d2-41e0-ad4e-ca761c116732-iptables-alerter-script\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.291191 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-log-socket\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.291232 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8180fada-f9bd-41d0-943e-70df65e3e2cd-konnectivity-ca\") pod \"konnectivity-agent-44d5g\" (UID: \"8180fada-f9bd-41d0-943e-70df65e3e2cd\") " pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.291342 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.291383 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291368 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 13:19:20.291776 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-host\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.291814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291797 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hcrs9\"" Jul 17 13:19:20.291852 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291821 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-node-log\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.291893 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291867 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-socket-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.291973 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.291956 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-device-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.292171 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292151 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.292212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292201 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff48e2f8-23d2-41e0-ad4e-ca761c116732-host-slash\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.292312 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292278 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-cni-bin\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.292378 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292343 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.292433 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292415 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovnkube-script-lib\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.292481 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292463 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-registration-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.292531 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtvf4\" (UniqueName: \"kubernetes.io/projected/a625f32a-cfe2-4918-9abe-1cda781faf78-kube-api-access-vtvf4\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.292645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-systemd\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.292699 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292661 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-modprobe-d\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.292752 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292730 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-systemd\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.292853 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292833 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.293017 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293097 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.292923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zczvr\" (UniqueName: \"kubernetes.io/projected/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-kube-api-access-zczvr\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-systemd-units\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293348 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293437 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6nbz\" (UniqueName: \"kubernetes.io/projected/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-kube-api-access-b6nbz\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysctl-conf\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.293519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293513 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:20.293975 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293542 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-etc-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.293975 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.293975 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293704 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6hzg\" (UniqueName: \"kubernetes.io/projected/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-kube-api-access-s6hzg\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.293975 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293758 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-run-netns\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.293975 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovnkube-config\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.293975 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.293928 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovn-node-metrics-cert\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.294385 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.294089 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8180fada-f9bd-41d0-943e-70df65e3e2cd-agent-certs\") pod \"konnectivity-agent-44d5g\" (UID: \"8180fada-f9bd-41d0-943e-70df65e3e2cd\") " pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.294870 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.294838 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 13:19:20.308966 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.308946 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lvgpz" Jul 17 13:19:20.314539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.314522 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lvgpz" Jul 17 13:19:20.380819 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.380771 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 13:19:20.394609 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6hzg\" (UniqueName: \"kubernetes.io/projected/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-kube-api-access-s6hzg\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.394660 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394614 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovnkube-config\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.394660 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovn-node-metrics-cert\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.394720 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394661 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8180fada-f9bd-41d0-943e-70df65e3e2cd-agent-certs\") pod \"konnectivity-agent-44d5g\" (UID: \"8180fada-f9bd-41d0-943e-70df65e3e2cd\") " pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.394720 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-kubernetes\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.394844 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394761 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-kubernetes\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.394844 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394785 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysctl-d\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.394844 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394820 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-sys\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.395017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394847 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-cni-netd\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394890 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-sys-fs\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.395017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-multus-certs\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.395017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394961 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysctl-d\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.395017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.394982 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-env-overrides\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395022 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-sys-fs\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.395225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395008 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.395225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395031 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-cni-netd\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395061 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff48e2f8-23d2-41e0-ad4e-ca761c116732-iptables-alerter-script\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.395225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395096 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8180fada-f9bd-41d0-943e-70df65e3e2cd-konnectivity-ca\") pod \"konnectivity-agent-44d5g\" (UID: \"8180fada-f9bd-41d0-943e-70df65e3e2cd\") " pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.395225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395176 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-sys\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.395225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395214 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-socket-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395329 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a463f487-d248-42f7-8745-f3b1e6869b9f-hosts-file\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395390 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-socket-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-system-cni-dir\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-node-log\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395476 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-env-overrides\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395507 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-device-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395500 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovnkube-config\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-cni-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395577 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-node-log\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395621 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-cni-multus\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395650 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-device-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-cni-bin\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395746 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395782 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovnkube-script-lib\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395792 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-host\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-systemd\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395841 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8180fada-f9bd-41d0-943e-70df65e3e2cd-konnectivity-ca\") pod \"konnectivity-agent-44d5g\" (UID: \"8180fada-f9bd-41d0-943e-70df65e3e2cd\") " pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395808 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ff48e2f8-23d2-41e0-ad4e-ca761c116732-iptables-alerter-script\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-cni-bin\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395887 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-systemd\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395937 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6nbz\" (UniqueName: \"kubernetes.io/projected/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-kube-api-access-b6nbz\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.395999 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.395980 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-systemd\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396037 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-systemd\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysctl-conf\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-etc-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396149 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-system-cni-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396182 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-cnibin\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396243 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-socket-dir-parent\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396265 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysctl-conf\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396269 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-os-release\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b91e071d-432f-4a3d-8f62-f18403981b2d-kubelet-config\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396314 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-etc-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396344 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-run-netns\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-serviceca\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396420 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-run\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396448 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-run-netns\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-slash\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396536 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-slash\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.396650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l59mz\" (UniqueName: \"kubernetes.io/projected/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-kube-api-access-l59mz\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396596 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a463f487-d248-42f7-8745-f3b1e6869b9f-tmp-dir\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396674 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-run\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396663 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-var-lib-kubelet\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396722 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-var-lib-kubelet\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396752 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-tuned\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396769 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkc9x\" (UniqueName: \"kubernetes.io/projected/ff48e2f8-23d2-41e0-ad4e-ca761c116732-kube-api-access-pkc9x\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-kubelet\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-var-lib-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-kubelet\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396854 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-var-lib-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396872 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovnkube-script-lib\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396863 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-ovn\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396932 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-openvswitch\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396956 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-run-ovn\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.397457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.396932 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397025 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-kubelet-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397052 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysconfig\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-lib-modules\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397119 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-tmp\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397129 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-sysconfig\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-log-socket\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397251 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-etc-selinux\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397273 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-log-socket\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-registration-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397307 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-lib-modules\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-os-release\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397353 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-host\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397368 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a625f32a-cfe2-4918-9abe-1cda781faf78-registration-dir\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397378 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-k8s-cni-cncf-io\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397404 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-netns\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-kubelet\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-conf-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-etc-kubernetes\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397476 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-host\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397534 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dbtd\" (UniqueName: \"kubernetes.io/projected/a463f487-d248-42f7-8745-f3b1e6869b9f-kube-api-access-6dbtd\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff48e2f8-23d2-41e0-ad4e-ca761c116732-host-slash\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtvf4\" (UniqueName: \"kubernetes.io/projected/a625f32a-cfe2-4918-9abe-1cda781faf78-kube-api-access-vtvf4\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397645 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff48e2f8-23d2-41e0-ad4e-ca761c116732-host-slash\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397685 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-hostroot\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-daemon-config\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397806 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397829 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-modprobe-d\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397855 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zczvr\" (UniqueName: \"kubernetes.io/projected/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-kube-api-access-zczvr\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-systemd-units\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397923 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-cnibin\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.397995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnk67\" (UniqueName: \"kubernetes.io/projected/99db032f-ab0a-405e-9482-74bd20a526f1-kube-api-access-nnk67\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.398991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398026 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-modprobe-d\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398042 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bt4s\" (UniqueName: \"kubernetes.io/projected/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-kube-api-access-4bt4s\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398053 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-systemd-units\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398143 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398186 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-cni-binary-copy\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398216 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-cni-bin\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.398326 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398332 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-host-run-ovn-kubernetes\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.398350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b91e071d-432f-4a3d-8f62-f18403981b2d-dbus\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.398439 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs podName:1992cdcc-16ba-4ebe-a0a8-b382deed8fcb nodeName:}" failed. No retries permitted until 2026-07-17 13:19:20.898411304 +0000 UTC m=+2.976524947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs") pod "network-metrics-daemon-pfz9l" (UID: "1992cdcc-16ba-4ebe-a0a8-b382deed8fcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.399070 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-ovn-node-metrics-cert\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.399112 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8180fada-f9bd-41d0-943e-70df65e3e2cd-agent-certs\") pod \"konnectivity-agent-44d5g\" (UID: \"8180fada-f9bd-41d0-943e-70df65e3e2cd\") " pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.399200 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-etc-tuned\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.399552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.399300 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-tmp\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.402038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.402021 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6hzg\" (UniqueName: \"kubernetes.io/projected/6f5bfaeb-5c9c-4b0b-98e5-c27acb371162-kube-api-access-s6hzg\") pod \"tuned-m5nr4\" (UID: \"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162\") " pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.403568 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.403546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6nbz\" (UniqueName: \"kubernetes.io/projected/e3b450f3-ffc7-4ea6-8283-7772fae77cbc-kube-api-access-b6nbz\") pod \"ovnkube-node-s478r\" (UID: \"e3b450f3-ffc7-4ea6-8283-7772fae77cbc\") " pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.404052 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.404032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkc9x\" (UniqueName: \"kubernetes.io/projected/ff48e2f8-23d2-41e0-ad4e-ca761c116732-kube-api-access-pkc9x\") pod \"iptables-alerter-wxgng\" (UID: \"ff48e2f8-23d2-41e0-ad4e-ca761c116732\") " pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.406444 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.406426 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtvf4\" (UniqueName: \"kubernetes.io/projected/a625f32a-cfe2-4918-9abe-1cda781faf78-kube-api-access-vtvf4\") pod \"aws-ebs-csi-driver-node-pkjlv\" (UID: \"a625f32a-cfe2-4918-9abe-1cda781faf78\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.408723 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.406991 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zczvr\" (UniqueName: \"kubernetes.io/projected/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-kube-api-access-zczvr\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:20.459254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.459228 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 13:19:20.499775 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499739 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-os-release\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.499965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499859 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-os-release\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.499965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499865 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-k8s-cni-cncf-io\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.499965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499901 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-netns\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.499965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499940 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-kubelet\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.499965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499941 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-k8s-cni-cncf-io\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.499965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499961 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-conf-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499983 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-etc-kubernetes\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.499993 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-netns\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500006 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500031 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dbtd\" (UniqueName: \"kubernetes.io/projected/a463f487-d248-42f7-8745-f3b1e6869b9f-kube-api-access-6dbtd\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500055 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-hostroot\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500040 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-conf-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500079 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-daemon-config\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500107 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-kubelet\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500132 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-cnibin\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnk67\" (UniqueName: \"kubernetes.io/projected/99db032f-ab0a-405e-9482-74bd20a526f1-kube-api-access-nnk67\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bt4s\" (UniqueName: \"kubernetes.io/projected/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-kube-api-access-4bt4s\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500219 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-cni-binary-copy\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-cni-bin\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b91e071d-432f-4a3d-8f62-f18403981b2d-dbus\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-multus-certs\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500390 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a463f487-d248-42f7-8745-f3b1e6869b9f-hosts-file\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-system-cni-dir\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500422 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-etc-kubernetes\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-cni-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-cni-multus\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500489 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-hostroot\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500543 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-host\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-system-cni-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500598 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-cnibin\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-socket-dir-parent\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500644 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-os-release\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500669 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b91e071d-432f-4a3d-8f62-f18403981b2d-kubelet-config\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500694 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-serviceca\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.500954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l59mz\" (UniqueName: \"kubernetes.io/projected/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-kube-api-access-l59mz\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a463f487-d248-42f7-8745-f3b1e6869b9f-tmp-dir\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.500782 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501092 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-daemon-config\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.501199 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501228 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-cni-bin\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501248 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-run-multus-certs\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.501267 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret podName:b91e071d-432f-4a3d-8f62-f18403981b2d nodeName:}" failed. No retries permitted until 2026-07-17 13:19:21.001250065 +0000 UTC m=+3.079363711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret") pod "global-pull-secret-syncer-9b6c9" (UID: "b91e071d-432f-4a3d-8f62-f18403981b2d") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501291 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-cnibin\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501375 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501401 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501403 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b91e071d-432f-4a3d-8f62-f18403981b2d-dbus\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a463f487-d248-42f7-8745-f3b1e6869b9f-hosts-file\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501515 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-system-cni-dir\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501575 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-cni-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501613 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-host-var-lib-cni-multus\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501679 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b91e071d-432f-4a3d-8f62-f18403981b2d-kubelet-config\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:20.501874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99db032f-ab0a-405e-9482-74bd20a526f1-os-release\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501926 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.501996 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-system-cni-dir\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.502035 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-host\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.502065 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-cni-binary-copy\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.502132 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-cnibin\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.502191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-serviceca\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.502191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99db032f-ab0a-405e-9482-74bd20a526f1-cni-binary-copy\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.502212 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-multus-socket-dir-parent\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.502677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.502424 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a463f487-d248-42f7-8745-f3b1e6869b9f-tmp-dir\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.507463 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.507042 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:19:20.507463 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.507067 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:19:20.507463 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.507080 2560 projected.go:196] Error preparing data for projected volume kube-api-access-7l9pd for pod openshift-network-diagnostics/network-check-target-8nhwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:20.507463 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.507131 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd podName:2441c8b2-c301-47f9-96f2-eb0049a7dda3 nodeName:}" failed. No retries permitted until 2026-07-17 13:19:21.007116517 +0000 UTC m=+3.085230165 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7l9pd" (UniqueName: "kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd") pod "network-check-target-8nhwn" (UID: "2441c8b2-c301-47f9-96f2-eb0049a7dda3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:20.509314 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.509289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bt4s\" (UniqueName: \"kubernetes.io/projected/28f66006-4ac8-4413-bd8c-e355c9cb4c8c-kube-api-access-4bt4s\") pod \"node-ca-twxnh\" (UID: \"28f66006-4ac8-4413-bd8c-e355c9cb4c8c\") " pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.509779 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.509744 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnk67\" (UniqueName: \"kubernetes.io/projected/99db032f-ab0a-405e-9482-74bd20a526f1-kube-api-access-nnk67\") pod \"multus-additional-cni-plugins-lv575\" (UID: \"99db032f-ab0a-405e-9482-74bd20a526f1\") " pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.510074 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.510032 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l59mz\" (UniqueName: \"kubernetes.io/projected/7d1bb89e-f6cc-4f03-a214-4ba879a513d0-kube-api-access-l59mz\") pod \"multus-t2xx7\" (UID: \"7d1bb89e-f6cc-4f03-a214-4ba879a513d0\") " pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.511344 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.511291 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dbtd\" (UniqueName: \"kubernetes.io/projected/a463f487-d248-42f7-8745-f3b1e6869b9f-kube-api-access-6dbtd\") pod \"node-resolver-52ncl\" (UID: \"a463f487-d248-42f7-8745-f3b1e6869b9f\") " pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.584517 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.584487 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" Jul 17 13:19:20.594366 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.594165 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wxgng" Jul 17 13:19:20.595559 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:20.595430 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5bfaeb_5c9c_4b0b_98e5_c27acb371162.slice/crio-fce22521780af05e81df459a41baecada2d82f231f8d0def1e1129e78148ab73 WatchSource:0}: Error finding container fce22521780af05e81df459a41baecada2d82f231f8d0def1e1129e78148ab73: Status 404 returned error can't find the container with id fce22521780af05e81df459a41baecada2d82f231f8d0def1e1129e78148ab73 Jul 17 13:19:20.601723 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.601572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:20.605104 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.605089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:20.611415 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.611311 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" Jul 17 13:19:20.617243 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:20.617014 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8180fada_f9bd_41d0_943e_70df65e3e2cd.slice/crio-f3806f312672c4a2fd242e50759ebac71585c416d558598b84a452b1d07d6de6 WatchSource:0}: Error finding container f3806f312672c4a2fd242e50759ebac71585c416d558598b84a452b1d07d6de6: Status 404 returned error can't find the container with id f3806f312672c4a2fd242e50759ebac71585c416d558598b84a452b1d07d6de6 Jul 17 13:19:20.617243 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.617087 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-52ncl" Jul 17 13:19:20.621035 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:20.621012 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b450f3_ffc7_4ea6_8283_7772fae77cbc.slice/crio-709e3ec776c6526ee7bcbba45068462f928e879f53fdffac2e3fd8d7fa148aab WatchSource:0}: Error finding container 709e3ec776c6526ee7bcbba45068462f928e879f53fdffac2e3fd8d7fa148aab: Status 404 returned error can't find the container with id 709e3ec776c6526ee7bcbba45068462f928e879f53fdffac2e3fd8d7fa148aab Jul 17 13:19:20.623038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.622924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-twxnh" Jul 17 13:19:20.628925 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.628892 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lv575" Jul 17 13:19:20.636090 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:20.636063 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda625f32a_cfe2_4918_9abe_1cda781faf78.slice/crio-357a879363113e07f5bf20a76c0b03112fb2f7e85d7a4fd51065bb5092ad4a06 WatchSource:0}: Error finding container 357a879363113e07f5bf20a76c0b03112fb2f7e85d7a4fd51065bb5092ad4a06: Status 404 returned error can't find the container with id 357a879363113e07f5bf20a76c0b03112fb2f7e85d7a4fd51065bb5092ad4a06 Jul 17 13:19:20.636293 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.636277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t2xx7" Jul 17 13:19:20.650373 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:19:20.650349 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d1bb89e_f6cc_4f03_a214_4ba879a513d0.slice/crio-b717f3b75bd50a9cb70a7e4dde6874891bda6c61e5ff1d83a5e170ed3a7ed968 WatchSource:0}: Error finding container b717f3b75bd50a9cb70a7e4dde6874891bda6c61e5ff1d83a5e170ed3a7ed968: Status 404 returned error can't find the container with id b717f3b75bd50a9cb70a7e4dde6874891bda6c61e5ff1d83a5e170ed3a7ed968 Jul 17 13:19:20.903982 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:20.903857 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:20.904132 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.904008 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:20.904132 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:20.904079 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs podName:1992cdcc-16ba-4ebe-a0a8-b382deed8fcb nodeName:}" failed. No retries permitted until 2026-07-17 13:19:21.904057653 +0000 UTC m=+3.982171297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs") pod "network-metrics-daemon-pfz9l" (UID: "1992cdcc-16ba-4ebe-a0a8-b382deed8fcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:21.004327 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.004268 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:21.004497 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.004394 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:21.004497 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.004481 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret podName:b91e071d-432f-4a3d-8f62-f18403981b2d nodeName:}" failed. No retries permitted until 2026-07-17 13:19:22.004459927 +0000 UTC m=+4.082573587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret") pod "global-pull-secret-syncer-9b6c9" (UID: "b91e071d-432f-4a3d-8f62-f18403981b2d") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:21.103249 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.103204 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerStarted","Data":"422536e27f8f32e3028982fc0e1d0246668c8110a73275fe499049abbeb1d03d"} Jul 17 13:19:21.104946 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.104808 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:21.105079 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.105028 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:19:21.105079 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.105050 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:19:21.105079 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.105063 2560 projected.go:196] Error preparing data for projected volume kube-api-access-7l9pd for pod openshift-network-diagnostics/network-check-target-8nhwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:21.105259 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.105127 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd podName:2441c8b2-c301-47f9-96f2-eb0049a7dda3 nodeName:}" failed. No retries permitted until 2026-07-17 13:19:22.105109462 +0000 UTC m=+4.183223126 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l9pd" (UniqueName: "kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd") pod "network-check-target-8nhwn" (UID: "2441c8b2-c301-47f9-96f2-eb0049a7dda3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:21.105525 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.105496 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-twxnh" event={"ID":"28f66006-4ac8-4413-bd8c-e355c9cb4c8c","Type":"ContainerStarted","Data":"7c8b5a96918e87a927d36f4bb290f7e17e768892ed18cb569f439666aa8f5c7c"} Jul 17 13:19:21.106955 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.106928 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"709e3ec776c6526ee7bcbba45068462f928e879f53fdffac2e3fd8d7fa148aab"} Jul 17 13:19:21.109486 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.109461 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wxgng" event={"ID":"ff48e2f8-23d2-41e0-ad4e-ca761c116732","Type":"ContainerStarted","Data":"35d912fcf1ffcb819858c86cf8f090a44729c176cf809d3a86f36c074e992024"} Jul 17 13:19:21.111247 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.111195 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" event={"ID":"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162","Type":"ContainerStarted","Data":"fce22521780af05e81df459a41baecada2d82f231f8d0def1e1129e78148ab73"} Jul 17 13:19:21.112656 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.112592 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t2xx7" event={"ID":"7d1bb89e-f6cc-4f03-a214-4ba879a513d0","Type":"ContainerStarted","Data":"b717f3b75bd50a9cb70a7e4dde6874891bda6c61e5ff1d83a5e170ed3a7ed968"} Jul 17 13:19:21.114118 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.114048 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-52ncl" event={"ID":"a463f487-d248-42f7-8745-f3b1e6869b9f","Type":"ContainerStarted","Data":"cecbd5383077d882796ef8992e0dd90079fce2e59566dff13d8b95bd0048f564"} Jul 17 13:19:21.116085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.116000 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" event={"ID":"a625f32a-cfe2-4918-9abe-1cda781faf78","Type":"ContainerStarted","Data":"357a879363113e07f5bf20a76c0b03112fb2f7e85d7a4fd51065bb5092ad4a06"} Jul 17 13:19:21.117968 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.117946 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-44d5g" event={"ID":"8180fada-f9bd-41d0-943e-70df65e3e2cd","Type":"ContainerStarted","Data":"f3806f312672c4a2fd242e50759ebac71585c416d558598b84a452b1d07d6de6"} Jul 17 13:19:21.316655 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.316611 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 13:14:20 +0000 UTC" deadline="2028-03-28 12:16:57.063527012 +0000 UTC" Jul 17 13:19:21.316655 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.316650 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14878h57m35.746880916s" Jul 17 13:19:21.911967 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:21.911836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:21.912688 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.912024 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:21.912688 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:21.912090 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs podName:1992cdcc-16ba-4ebe-a0a8-b382deed8fcb nodeName:}" failed. No retries permitted until 2026-07-17 13:19:23.912072598 +0000 UTC m=+5.990186257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs") pod "network-metrics-daemon-pfz9l" (UID: "1992cdcc-16ba-4ebe-a0a8-b382deed8fcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:22.013829 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:22.013546 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:22.013829 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.013727 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:22.013829 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.013791 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret podName:b91e071d-432f-4a3d-8f62-f18403981b2d nodeName:}" failed. No retries permitted until 2026-07-17 13:19:24.013772689 +0000 UTC m=+6.091886354 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret") pod "global-pull-secret-syncer-9b6c9" (UID: "b91e071d-432f-4a3d-8f62-f18403981b2d") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:22.096395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:22.095902 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:22.096395 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.096043 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:22.096395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:22.096135 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:22.096395 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.096223 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:22.096395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:22.096284 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:22.096395 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.096353 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:22.114509 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:22.114260 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:22.114509 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.114397 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:19:22.114509 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.114414 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:19:22.114509 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.114426 2560 projected.go:196] Error preparing data for projected volume kube-api-access-7l9pd for pod openshift-network-diagnostics/network-check-target-8nhwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:22.114509 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:22.114475 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd podName:2441c8b2-c301-47f9-96f2-eb0049a7dda3 nodeName:}" failed. No retries permitted until 2026-07-17 13:19:24.114458533 +0000 UTC m=+6.192572195 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l9pd" (UniqueName: "kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd") pod "network-check-target-8nhwn" (UID: "2441c8b2-c301-47f9-96f2-eb0049a7dda3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:22.317569 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:22.317504 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 13:14:20 +0000 UTC" deadline="2028-01-15 18:31:31.278902049 +0000 UTC" Jul 17 13:19:22.317569 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:22.317566 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13133h12m8.961339963s" Jul 17 13:19:23.928239 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:23.928208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:23.928721 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:23.928358 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:23.928721 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:23.928430 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs podName:1992cdcc-16ba-4ebe-a0a8-b382deed8fcb nodeName:}" failed. No retries permitted until 2026-07-17 13:19:27.928410714 +0000 UTC m=+10.006524362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs") pod "network-metrics-daemon-pfz9l" (UID: "1992cdcc-16ba-4ebe-a0a8-b382deed8fcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:24.029199 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:24.029156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:24.029359 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.029311 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:24.029427 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.029372 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret podName:b91e071d-432f-4a3d-8f62-f18403981b2d nodeName:}" failed. No retries permitted until 2026-07-17 13:19:28.029351779 +0000 UTC m=+10.107465443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret") pod "global-pull-secret-syncer-9b6c9" (UID: "b91e071d-432f-4a3d-8f62-f18403981b2d") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:24.095445 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:24.095410 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:24.095605 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:24.095461 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:24.095605 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:24.095410 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:24.095605 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.095544 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:24.095771 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.095694 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:24.095823 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.095787 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:24.130163 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:24.130133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:24.130314 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.130299 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:19:24.130395 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.130322 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:19:24.130395 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.130335 2560 projected.go:196] Error preparing data for projected volume kube-api-access-7l9pd for pod openshift-network-diagnostics/network-check-target-8nhwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:24.130395 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:24.130391 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd podName:2441c8b2-c301-47f9-96f2-eb0049a7dda3 nodeName:}" failed. No retries permitted until 2026-07-17 13:19:28.130372669 +0000 UTC m=+10.208486326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l9pd" (UniqueName: "kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd") pod "network-check-target-8nhwn" (UID: "2441c8b2-c301-47f9-96f2-eb0049a7dda3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:26.095572 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:26.095478 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:26.096035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:26.095489 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:26.096035 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:26.095606 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:26.096035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:26.095672 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:26.096035 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:26.095773 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:26.096035 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:26.095854 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:27.959621 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:27.959579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:27.960090 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:27.959789 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:27.960090 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:27.959856 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs podName:1992cdcc-16ba-4ebe-a0a8-b382deed8fcb nodeName:}" failed. No retries permitted until 2026-07-17 13:19:35.959838447 +0000 UTC m=+18.037952090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs") pod "network-metrics-daemon-pfz9l" (UID: "1992cdcc-16ba-4ebe-a0a8-b382deed8fcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:28.060228 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:28.060189 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:28.060404 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.060362 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:28.060461 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.060439 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret podName:b91e071d-432f-4a3d-8f62-f18403981b2d nodeName:}" failed. No retries permitted until 2026-07-17 13:19:36.060417522 +0000 UTC m=+18.138531167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret") pod "global-pull-secret-syncer-9b6c9" (UID: "b91e071d-432f-4a3d-8f62-f18403981b2d") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:28.095328 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:28.095251 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:28.095328 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:28.095269 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:28.095328 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:28.095277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:28.095588 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.095369 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:28.095588 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.095446 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:28.095588 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.095541 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:28.161184 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:28.161134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:28.161334 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.161307 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:19:28.161334 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.161329 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:19:28.161447 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.161341 2560 projected.go:196] Error preparing data for projected volume kube-api-access-7l9pd for pod openshift-network-diagnostics/network-check-target-8nhwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:28.161447 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:28.161403 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd podName:2441c8b2-c301-47f9-96f2-eb0049a7dda3 nodeName:}" failed. No retries permitted until 2026-07-17 13:19:36.16138379 +0000 UTC m=+18.239497434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l9pd" (UniqueName: "kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd") pod "network-check-target-8nhwn" (UID: "2441c8b2-c301-47f9-96f2-eb0049a7dda3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:30.095853 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:30.095515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:30.095853 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:30.095559 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:30.095853 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:30.095523 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:30.095853 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:30.095687 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:30.095853 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:30.095726 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:30.095853 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:30.095811 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:32.096018 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:32.095984 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:32.096470 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:32.096121 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:32.096470 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:32.095991 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:32.096470 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:32.095985 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:32.096470 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:32.096237 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:32.096470 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:32.096315 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:34.096233 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:34.096197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:34.096233 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:34.096197 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:34.096692 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:34.096224 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:34.096692 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:34.096307 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:34.096692 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:34.096411 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:34.096692 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:34.096491 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:36.019989 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:36.019791 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:36.020459 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.019970 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:36.020459 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.020069 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs podName:1992cdcc-16ba-4ebe-a0a8-b382deed8fcb nodeName:}" failed. No retries permitted until 2026-07-17 13:19:52.020046992 +0000 UTC m=+34.098160647 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs") pod "network-metrics-daemon-pfz9l" (UID: "1992cdcc-16ba-4ebe-a0a8-b382deed8fcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:36.095891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:36.095854 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:36.095891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:36.095893 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:36.096098 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:36.095869 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:36.096098 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.095997 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:36.096191 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.096125 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:36.096231 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.096214 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:36.120488 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:36.120458 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:36.120630 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.120587 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:36.120701 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.120633 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret podName:b91e071d-432f-4a3d-8f62-f18403981b2d nodeName:}" failed. No retries permitted until 2026-07-17 13:19:52.120620533 +0000 UTC m=+34.198734175 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret") pod "global-pull-secret-syncer-9b6c9" (UID: "b91e071d-432f-4a3d-8f62-f18403981b2d") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:36.221486 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:36.221451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:36.221673 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.221576 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:19:36.221673 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.221591 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:19:36.221673 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.221599 2560 projected.go:196] Error preparing data for projected volume kube-api-access-7l9pd for pod openshift-network-diagnostics/network-check-target-8nhwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:36.221673 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:36.221645 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd podName:2441c8b2-c301-47f9-96f2-eb0049a7dda3 nodeName:}" failed. No retries permitted until 2026-07-17 13:19:52.221631121 +0000 UTC m=+34.299744763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l9pd" (UniqueName: "kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd") pod "network-check-target-8nhwn" (UID: "2441c8b2-c301-47f9-96f2-eb0049a7dda3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:38.095896 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:38.095866 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:38.096294 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:38.095868 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:38.096294 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:38.095991 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:38.096294 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:38.095870 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:38.096294 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:38.096090 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:38.096294 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:38.096171 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:39.153645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.153348 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerStarted","Data":"b768b5aff26ea6b97352a6bd8ab5fef02afe3fec0c6c7a5ff66c396ea550669c"} Jul 17 13:19:39.156254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.156106 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-twxnh" event={"ID":"28f66006-4ac8-4413-bd8c-e355c9cb4c8c","Type":"ContainerStarted","Data":"90a064a769829c6640a38aa3372bf5835e2d9657bbb63d4cdf708f56a6ec2a13"} Jul 17 13:19:39.158420 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.158394 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"d58ce9c7f8fec8f8a31b0ef104d9f45e16d59a7aab8c4b563eeac5bc95c7439d"} Jul 17 13:19:39.158597 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.158580 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"93f31adaceaa2d211dd2728f3f1b63ed7292cb968230af2bb8eb1dda2cd1921c"} Jul 17 13:19:39.158745 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.158715 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"1791b914980826c239675104e32b246e1785c3dfd846260bd227b65fc04b239e"} Jul 17 13:19:39.160024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.159980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" event={"ID":"6f5bfaeb-5c9c-4b0b-98e5-c27acb371162","Type":"ContainerStarted","Data":"be2a353ac6bc7b47c79dafb2714e1f565c1fa3ce26152ad4bd4b3badc81f9bbb"} Jul 17 13:19:39.161632 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.161609 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t2xx7" event={"ID":"7d1bb89e-f6cc-4f03-a214-4ba879a513d0","Type":"ContainerStarted","Data":"f46e8c2bd153fe27be8bde0fb53b4f04c58e81513db93aa82e7808a7dc000c5b"} Jul 17 13:19:39.163446 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.163364 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-52ncl" event={"ID":"a463f487-d248-42f7-8745-f3b1e6869b9f","Type":"ContainerStarted","Data":"bc87acf4ea34b5d1a1aa91f991760674ab0d2d62df23fd8b396310aa735a8c35"} Jul 17 13:19:39.165187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.165154 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" event={"ID":"a625f32a-cfe2-4918-9abe-1cda781faf78","Type":"ContainerStarted","Data":"9c7e588bfb318009fa39ff0b6a4aa1847b599ce0f328678dbf75c00494c72f69"} Jul 17 13:19:39.166623 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.166603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-44d5g" event={"ID":"8180fada-f9bd-41d0-943e-70df65e3e2cd","Type":"ContainerStarted","Data":"db811553c96e8ca7b1945e8053d0a7744786886b041ae16069ee138933bea9bd"} Jul 17 13:19:39.168354 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.168095 2560 generic.go:356] "Generic (PLEG): container finished" podID="fe62c43e5ee9d2052903089d10f2a507" containerID="561b21e9d9f44cd768f71de687404c524dbba8eae8f672c3c087b4a8fa080584" exitCode=0 Jul 17 13:19:39.168354 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.168162 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" event={"ID":"fe62c43e5ee9d2052903089d10f2a507","Type":"ContainerDied","Data":"561b21e9d9f44cd768f71de687404c524dbba8eae8f672c3c087b4a8fa080584"} Jul 17 13:19:39.170202 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.170185 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" event={"ID":"068abd2681734d05200c5d2715b73db9","Type":"ContainerStarted","Data":"e8aadbefa293f28be925597897335fc76840dddce5776bee57e5a906eedbbdec"} Jul 17 13:19:39.187706 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.187668 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-52ncl" podStartSLOduration=2.334674733 podStartE2EDuration="20.187648263s" totalImagesPullingTime="17.85297353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:39.18575123 +0000 UTC m=+21.263864895" watchObservedRunningTime="2026-07-17 13:19:39.187648263 +0000 UTC m=+21.265761926" Jul 17 13:19:39.201846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.201814 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t2xx7" podStartSLOduration=2.025341997 podStartE2EDuration="20.201802963s" totalImagesPullingTime="18.176460966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:39.201237402 +0000 UTC m=+21.279351078" watchObservedRunningTime="2026-07-17 13:19:39.201802963 +0000 UTC m=+21.279916626" Jul 17 13:19:39.218626 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.218594 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-189.ec2.internal" podStartSLOduration=20.218585135 podStartE2EDuration="20.218585135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:39.218494499 +0000 UTC m=+21.296608163" watchObservedRunningTime="2026-07-17 13:19:39.218585135 +0000 UTC m=+21.296698799" Jul 17 13:19:39.239670 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.239639 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m5nr4" podStartSLOduration=2.141840392 podStartE2EDuration="20.239629149s" totalImagesPullingTime="18.097788757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:39.239219251 +0000 UTC m=+21.317332916" watchObservedRunningTime="2026-07-17 13:19:39.239629149 +0000 UTC m=+21.317742809" Jul 17 13:19:39.254851 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.254815 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-twxnh" podStartSLOduration=10.628302846 podStartE2EDuration="20.254805649s" totalImagesPullingTime="9.626502803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:39.254422739 +0000 UTC m=+21.332536406" watchObservedRunningTime="2026-07-17 13:19:39.254805649 +0000 UTC m=+21.332919314" Jul 17 13:19:39.268266 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.268232 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-44d5g" podStartSLOduration=10.618760603 podStartE2EDuration="20.268220944s" totalImagesPullingTime="9.649460341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:39.267836786 +0000 UTC m=+21.345950451" watchObservedRunningTime="2026-07-17 13:19:39.268220944 +0000 UTC m=+21.346334606" Jul 17 13:19:39.366228 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.366198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:39.450264 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.450232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:39.550341 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.550244 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:39.859272 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.859189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:39.912527 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.912500 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:39.927270 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:39.927255 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-52ncl_a463f487-d248-42f7-8745-f3b1e6869b9f/dns-node-resolver/0.log" Jul 17 13:19:40.095322 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.095301 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:40.095427 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.095324 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:40.095427 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.095324 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:40.095525 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:40.095425 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:40.095577 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:40.095537 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:40.095649 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:40.095629 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:40.176468 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.175574 2560 generic.go:356] "Generic (PLEG): container finished" podID="99db032f-ab0a-405e-9482-74bd20a526f1" containerID="b768b5aff26ea6b97352a6bd8ab5fef02afe3fec0c6c7a5ff66c396ea550669c" exitCode=0 Jul 17 13:19:40.176468 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.175700 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerDied","Data":"b768b5aff26ea6b97352a6bd8ab5fef02afe3fec0c6c7a5ff66c396ea550669c"} Jul 17 13:19:40.180112 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.180082 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"3a0556583f756a61033d3c6da1ca3f5efffe51370f361cfbe82677d2e4428861"} Jul 17 13:19:40.180225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.180162 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"f3a2855ec65d53c7298daae7f962f2588a5d7f5b337a3c9351e5c9a208051213"} Jul 17 13:19:40.180225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.180221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"d52ff1d730fc3c006c1e3784d1ef1f221d5f901f9c2f03ad02972df4bed4234e"} Jul 17 13:19:40.181661 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.181627 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wxgng" event={"ID":"ff48e2f8-23d2-41e0-ad4e-ca761c116732","Type":"ContainerStarted","Data":"4250973ce30c1f525fa485ec5af34dd94b4f3e887c8751d1735737d4ab5d0f6b"} Jul 17 13:19:40.223287 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.223236 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wxgng" podStartSLOduration=11.563306227 podStartE2EDuration="21.223218896s" totalImagesPullingTime="9.659912669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:40.222821027 +0000 UTC m=+22.300934690" watchObservedRunningTime="2026-07-17 13:19:40.223218896 +0000 UTC m=+22.301332560" Jul 17 13:19:40.340553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.340511 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:40.402813 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.402732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:40.462271 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.462246 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 13:19:40.540579 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.540547 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:40.641159 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.641120 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:40.860024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.859923 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:40.959866 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:40.959837 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:41.015976 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.015846 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T13:19:40.462266678Z","UUID":"d46482de-2b29-4ac0-9240-bc48303a9a91","Handler":null,"Name":"","Endpoint":""} Jul 17 13:19:41.017672 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.017637 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 13:19:41.017672 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.017671 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 13:19:41.127850 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.127765 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-twxnh_28f66006-4ac8-4413-bd8c-e355c9cb4c8c/node-ca/0.log" Jul 17 13:19:41.186262 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.186228 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" event={"ID":"fe62c43e5ee9d2052903089d10f2a507","Type":"ContainerStarted","Data":"dec177c33e4df596e8b69d82a7ceac72ba3f1c0044caa017b73f7a988a97b8de"} Jul 17 13:19:41.188980 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.188951 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" event={"ID":"a625f32a-cfe2-4918-9abe-1cda781faf78","Type":"ContainerStarted","Data":"ac8146fed569ee48f72edef14f0b973dc56ed2a701684a6b45f9130d762d7194"} Jul 17 13:19:41.554632 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.554597 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:41.814374 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.814336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:41.874417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.874389 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:41.875073 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.875051 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:41.892883 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:41.892835 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-189.ec2.internal" podStartSLOduration=22.892823296 podStartE2EDuration="22.892823296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:41.201329922 +0000 UTC m=+23.279443593" watchObservedRunningTime="2026-07-17 13:19:41.892823296 +0000 UTC m=+23.970936959" Jul 17 13:19:42.095967 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.095876 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:42.095967 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.095949 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:42.096139 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:42.096041 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:42.096139 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.096088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:42.096225 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:42.096152 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:42.096262 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:42.096218 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:42.194651 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.194617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"f08b9816368a170be3c30c536b7808fa5733bc828dcb4a01494a43b5de6324e3"} Jul 17 13:19:42.196797 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.196766 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" event={"ID":"a625f32a-cfe2-4918-9abe-1cda781faf78","Type":"ContainerStarted","Data":"9c2aea7fc5b8365a5c51acbafefa8dc07c74c6edb4a1234e5c36f38593e3e3ba"} Jul 17 13:19:42.197076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.197047 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:42.197540 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.197514 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-44d5g" Jul 17 13:19:42.215974 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.215935 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-pkjlv" podStartSLOduration=3.013872121 podStartE2EDuration="23.215918811s" totalImagesPullingTime="20.20204669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:42.214374748 +0000 UTC m=+24.292488412" watchObservedRunningTime="2026-07-17 13:19:42.215918811 +0000 UTC m=+24.294032491" Jul 17 13:19:42.709717 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.709683 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:42.767063 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:42.767018 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:44.096465 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:44.096275 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:44.096901 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:44.096346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:44.096901 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:44.096548 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:44.096901 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:44.096642 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:44.096901 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:44.096355 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:44.096901 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:44.096797 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:44.735898 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:44.735699 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:44.840011 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:44.839973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:44.936642 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:44.936617 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:45.049531 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.049495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:45.140628 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.140603 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:45.204481 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.204453 2560 generic.go:356] "Generic (PLEG): container finished" podID="99db032f-ab0a-405e-9482-74bd20a526f1" containerID="b8a3caa927f02e75f28a9d5dcb7b458e31323771d6283d6753479a3bd8f7f368" exitCode=0 Jul 17 13:19:45.204617 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.204518 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerDied","Data":"b8a3caa927f02e75f28a9d5dcb7b458e31323771d6283d6753479a3bd8f7f368"} Jul 17 13:19:45.208415 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.208397 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" event={"ID":"e3b450f3-ffc7-4ea6-8283-7772fae77cbc","Type":"ContainerStarted","Data":"b69929c3257cc0c4e754ef9faa15ac41c9adc85d46af8d06331382f60a8225d9"} Jul 17 13:19:45.208639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.208617 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:45.208698 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.208666 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:45.223752 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.223732 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:45.234537 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.234510 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:45.262010 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.261973 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" podStartSLOduration=8.088579279 podStartE2EDuration="26.261961015s" totalImagesPullingTime="18.173381736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:45.261407802 +0000 UTC m=+27.339521465" watchObservedRunningTime="2026-07-17 13:19:45.261961015 +0000 UTC m=+27.340074679" Jul 17 13:19:45.426047 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:45.426010 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:46.095240 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.095218 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:46.095338 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.095255 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:46.095338 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.095315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:46.095338 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:46.095326 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:46.095440 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:46.095384 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:46.095497 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:46.095469 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:46.212341 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.212308 2560 generic.go:356] "Generic (PLEG): container finished" podID="99db032f-ab0a-405e-9482-74bd20a526f1" containerID="5431d61a840ab1b0ca4f734528091ea81d1a4ae7ad708105b16ccda7daae5878" exitCode=0 Jul 17 13:19:46.212653 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.212368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerDied","Data":"5431d61a840ab1b0ca4f734528091ea81d1a4ae7ad708105b16ccda7daae5878"} Jul 17 13:19:46.212828 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.212801 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:46.228064 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.228045 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:19:46.430337 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.430298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:46.622837 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:46.622802 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:47.025355 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:47.025322 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:47.224059 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:47.224025 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:47.828838 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:47.828800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:48.032227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.032181 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:48.095414 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.095355 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:48.095504 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.095446 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:48.095504 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.095470 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:48.095564 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:48.095447 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:48.095564 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:48.095530 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:48.095628 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:48.095603 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:48.218325 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.218295 2560 generic.go:356] "Generic (PLEG): container finished" podID="99db032f-ab0a-405e-9482-74bd20a526f1" containerID="b8d6a810242998c0b436b90a1cf16cdf436d59408b747ec75feee15d8f39a5f5" exitCode=0 Jul 17 13:19:48.218426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.218374 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerDied","Data":"b8d6a810242998c0b436b90a1cf16cdf436d59408b747ec75feee15d8f39a5f5"} Jul 17 13:19:48.429599 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.429569 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:48.530189 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.530158 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:48.630786 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.630755 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:48.822853 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:48.822776 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:49.228122 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:49.228084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:49.329093 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:49.329053 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:49.856257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:49.856220 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:50.022570 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:50.022534 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:50.095670 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:50.095647 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:50.095846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:50.095648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:50.095846 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:50.095775 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:50.095846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:50.095794 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:50.095997 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:50.095892 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:50.096055 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:50.096000 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:50.627645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:50.627607 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:50.934955 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:50.934905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:51.038226 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:51.038195 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:51.424749 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:51.424713 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:51.824316 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:51.824136 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:52.041740 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.041705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:52.042212 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.041857 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:52.042212 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.041941 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs podName:1992cdcc-16ba-4ebe-a0a8-b382deed8fcb nodeName:}" failed. No retries permitted until 2026-07-17 13:20:24.041924173 +0000 UTC m=+66.120037833 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs") pod "network-metrics-daemon-pfz9l" (UID: "1992cdcc-16ba-4ebe-a0a8-b382deed8fcb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 13:19:52.095777 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.095711 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:52.095937 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.095711 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:52.095937 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.095833 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:52.095937 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.095708 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:52.096083 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.095969 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:52.096083 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.095995 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:52.142045 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.142016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:52.142203 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.142144 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:52.142203 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.142195 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret podName:b91e071d-432f-4a3d-8f62-f18403981b2d nodeName:}" failed. No retries permitted until 2026-07-17 13:20:24.142181207 +0000 UTC m=+66.220294852 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret") pod "global-pull-secret-syncer-9b6c9" (UID: "b91e071d-432f-4a3d-8f62-f18403981b2d") : object "kube-system"/"original-pull-secret" not registered Jul 17 13:19:52.231331 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.231295 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:52.243327 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.243301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:52.243462 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.243443 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 13:19:52.243516 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.243467 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 13:19:52.243516 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.243477 2560 projected.go:196] Error preparing data for projected volume kube-api-access-7l9pd for pod openshift-network-diagnostics/network-check-target-8nhwn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:52.243621 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:52.243529 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd podName:2441c8b2-c301-47f9-96f2-eb0049a7dda3 nodeName:}" failed. No retries permitted until 2026-07-17 13:20:24.24351114 +0000 UTC m=+66.321624781 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7l9pd" (UniqueName: "kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd") pod "network-check-target-8nhwn" (UID: "2441c8b2-c301-47f9-96f2-eb0049a7dda3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 13:19:52.424671 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:52.424634 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:53.027921 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:53.027867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:53.230640 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:53.230606 2560 generic.go:356] "Generic (PLEG): container finished" podID="99db032f-ab0a-405e-9482-74bd20a526f1" containerID="71cb2c44d53ed9bc9c770ef98a0e7aa44efc355db919e523621a47b77d7e123c" exitCode=0 Jul 17 13:19:53.231075 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:53.230669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerDied","Data":"71cb2c44d53ed9bc9c770ef98a0e7aa44efc355db919e523621a47b77d7e123c"} Jul 17 13:19:53.424105 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:53.424065 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:53.625061 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:53.624987 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:54.028650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:54.028620 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:54.095606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:54.095576 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:54.095716 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:54.095674 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:54.095716 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:54.095686 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:54.095716 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:54.095699 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:54.095877 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:54.095752 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:54.095877 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:54.095847 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:54.236250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:54.236213 2560 generic.go:356] "Generic (PLEG): container finished" podID="99db032f-ab0a-405e-9482-74bd20a526f1" containerID="6c288a0724ecad681ec91539c665e0ec316ce216fbdc22f24cb6238bc2ae72f8" exitCode=0 Jul 17 13:19:54.236702 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:54.236264 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerDied","Data":"6c288a0724ecad681ec91539c665e0ec316ce216fbdc22f24cb6238bc2ae72f8"} Jul 17 13:19:54.426536 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:54.426505 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:55.029401 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:55.029369 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:55.228044 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:55.228013 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:55.244423 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:55.244396 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lv575" event={"ID":"99db032f-ab0a-405e-9482-74bd20a526f1","Type":"ContainerStarted","Data":"45165ec207f0afb8d356ce55c7bd82e6aa20a17a3485b759e148c311fb1a3b7d"} Jul 17 13:19:55.267113 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:55.267073 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lv575" podStartSLOduration=15.375585076 podStartE2EDuration="36.267060192s" totalImagesPullingTime="20.891475116s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:19:55.266532639 +0000 UTC m=+37.344646303" watchObservedRunningTime="2026-07-17 13:19:55.267060192 +0000 UTC m=+37.345173855" Jul 17 13:19:55.421275 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:55.421244 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:56.095600 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:56.095572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:56.095756 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:56.095609 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:56.095756 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:56.095663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:56.095756 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:56.095674 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:56.095756 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:56.095735 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:56.095893 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:56.095786 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:19:57.714263 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:57.714228 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:57.762728 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:57.762702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:19:58.096175 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:58.096109 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:19:58.096175 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:58.096140 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:19:58.096378 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:58.096200 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:19:58.096378 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:19:58.096235 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:19:58.096378 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:58.096331 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:19:58.096541 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:19:58.096433 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:20:00.096276 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:00.096244 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:00.096276 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:00.096277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:00.096642 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:00.096244 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:00.096642 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:00.096339 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:20:00.096642 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:00.096399 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:20:00.096642 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:00.096462 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:20:02.095341 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.095310 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:02.095689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.095399 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:02.095689 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:02.095412 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:20:02.095689 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:02.095478 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:20:02.095689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.095486 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:02.095689 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:02.095570 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:20:02.342622 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.342589 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:02.443296 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.443263 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:02.752509 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.752428 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:02.841478 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.841441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:02.939072 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:02.939037 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:03.054951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.054868 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:03.139428 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.139385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:03.284930 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.284886 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:03.384721 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.384620 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:03.696932 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.696878 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:03.899623 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.899457 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:03.950510 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.950437 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9b6c9"] Jul 17 13:20:03.950654 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.950548 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:03.950654 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:03.950634 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:20:03.953645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.953619 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nhwn"] Jul 17 13:20:03.953782 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.953726 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:03.953845 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:03.953815 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:20:03.954255 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.954229 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pfz9l"] Jul 17 13:20:03.954356 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:03.954311 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:03.954434 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:03.954413 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:20:04.048346 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:04.048315 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:04.997383 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:04.997353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:05.054520 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:05.054478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:05.098115 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:05.098049 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:05.098243 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:05.098159 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:20:06.096088 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:06.096060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:06.096600 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:06.096066 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:06.096600 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:06.096148 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9b6c9" podUID="b91e071d-432f-4a3d-8f62-f18403981b2d" Jul 17 13:20:06.096600 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:06.096269 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pfz9l" podUID="1992cdcc-16ba-4ebe-a0a8-b382deed8fcb" Jul 17 13:20:07.095457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.095427 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:07.095698 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:07.095522 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8nhwn" podUID="2441c8b2-c301-47f9-96f2-eb0049a7dda3" Jul 17 13:20:07.671336 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.671309 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-189.ec2.internal" event="NodeReady" Jul 17 13:20:07.671749 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.671418 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 13:20:07.713480 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.713451 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5j8xf"] Jul 17 13:20:07.729598 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.729388 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xlfpb"] Jul 17 13:20:07.729598 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.729502 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.732449 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.732223 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 13:20:07.732449 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.732276 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 13:20:07.732449 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.732279 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 13:20:07.732449 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.732383 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-p7mfr\"" Jul 17 13:20:07.732449 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.732397 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 13:20:07.743320 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.743306 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5j8xf"] Jul 17 13:20:07.743390 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.743339 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xlfpb"] Jul 17 13:20:07.743425 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.743411 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:07.745971 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.745954 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rhkmk\"" Jul 17 13:20:07.746174 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.746159 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 13:20:07.746254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.746183 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 13:20:07.746254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.746199 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 13:20:07.760947 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.760925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f927fd7f-9e9a-4af5-8fd7-76c057df969d-cert\") pod \"ingress-canary-xlfpb\" (UID: \"f927fd7f-9e9a-4af5-8fd7-76c057df969d\") " pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:07.761035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.760965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3def5554-1674-4cfe-9da7-16f997510485-crio-socket\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.761035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.760995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3def5554-1674-4cfe-9da7-16f997510485-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.761194 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.761034 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw6fb\" (UniqueName: \"kubernetes.io/projected/3def5554-1674-4cfe-9da7-16f997510485-kube-api-access-hw6fb\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.761194 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.761088 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3def5554-1674-4cfe-9da7-16f997510485-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.761324 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.761194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf5xl\" (UniqueName: \"kubernetes.io/projected/f927fd7f-9e9a-4af5-8fd7-76c057df969d-kube-api-access-qf5xl\") pod \"ingress-canary-xlfpb\" (UID: \"f927fd7f-9e9a-4af5-8fd7-76c057df969d\") " pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:07.761324 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.761226 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def5554-1674-4cfe-9da7-16f997510485-data-volume\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.820001 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.819976 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9j845"] Jul 17 13:20:07.843293 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.843273 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.843293 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.843283 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9j845"] Jul 17 13:20:07.845735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.845719 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bdfvj\"" Jul 17 13:20:07.845976 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.845954 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 13:20:07.846115 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.846052 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 13:20:07.862475 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862451 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f927fd7f-9e9a-4af5-8fd7-76c057df969d-cert\") pod \"ingress-canary-xlfpb\" (UID: \"f927fd7f-9e9a-4af5-8fd7-76c057df969d\") " pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:07.862475 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862475 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3def5554-1674-4cfe-9da7-16f997510485-crio-socket\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.862597 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e65e5d8d-b568-4dea-a911-762096915096-config-volume\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.862597 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3def5554-1674-4cfe-9da7-16f997510485-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.862597 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862527 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e65e5d8d-b568-4dea-a911-762096915096-metrics-tls\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.862597 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862575 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65e5d8d-b568-4dea-a911-762096915096-tmp-dir\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.862778 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862616 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw6fb\" (UniqueName: \"kubernetes.io/projected/3def5554-1674-4cfe-9da7-16f997510485-kube-api-access-hw6fb\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.862778 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862652 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3def5554-1674-4cfe-9da7-16f997510485-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.862778 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862691 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8b86\" (UniqueName: \"kubernetes.io/projected/e65e5d8d-b568-4dea-a911-762096915096-kube-api-access-h8b86\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.862778 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862707 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3def5554-1674-4cfe-9da7-16f997510485-crio-socket\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.862778 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf5xl\" (UniqueName: \"kubernetes.io/projected/f927fd7f-9e9a-4af5-8fd7-76c057df969d-kube-api-access-qf5xl\") pod \"ingress-canary-xlfpb\" (UID: \"f927fd7f-9e9a-4af5-8fd7-76c057df969d\") " pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:07.862778 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.862756 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def5554-1674-4cfe-9da7-16f997510485-data-volume\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.863113 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.863094 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3def5554-1674-4cfe-9da7-16f997510485-data-volume\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.863113 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.863108 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3def5554-1674-4cfe-9da7-16f997510485-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.866340 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.866322 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3def5554-1674-4cfe-9da7-16f997510485-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.866457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.866443 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f927fd7f-9e9a-4af5-8fd7-76c057df969d-cert\") pod \"ingress-canary-xlfpb\" (UID: \"f927fd7f-9e9a-4af5-8fd7-76c057df969d\") " pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:07.871732 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.871714 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw6fb\" (UniqueName: \"kubernetes.io/projected/3def5554-1674-4cfe-9da7-16f997510485-kube-api-access-hw6fb\") pod \"insights-runtime-extractor-5j8xf\" (UID: \"3def5554-1674-4cfe-9da7-16f997510485\") " pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:07.871830 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.871767 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf5xl\" (UniqueName: \"kubernetes.io/projected/f927fd7f-9e9a-4af5-8fd7-76c057df969d-kube-api-access-qf5xl\") pod \"ingress-canary-xlfpb\" (UID: \"f927fd7f-9e9a-4af5-8fd7-76c057df969d\") " pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:07.917383 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.917353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:07.963708 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.963664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8b86\" (UniqueName: \"kubernetes.io/projected/e65e5d8d-b568-4dea-a911-762096915096-kube-api-access-h8b86\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.963790 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.963715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e65e5d8d-b568-4dea-a911-762096915096-config-volume\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.963790 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.963736 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e65e5d8d-b568-4dea-a911-762096915096-metrics-tls\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.963790 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.963765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65e5d8d-b568-4dea-a911-762096915096-tmp-dir\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.964071 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.964057 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e65e5d8d-b568-4dea-a911-762096915096-tmp-dir\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.964187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.964172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e65e5d8d-b568-4dea-a911-762096915096-config-volume\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.965856 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.965836 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e65e5d8d-b568-4dea-a911-762096915096-metrics-tls\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:07.976709 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:07.976688 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8b86\" (UniqueName: \"kubernetes.io/projected/e65e5d8d-b568-4dea-a911-762096915096-kube-api-access-h8b86\") pod \"dns-default-9j845\" (UID: \"e65e5d8d-b568-4dea-a911-762096915096\") " pod="openshift-dns/dns-default-9j845" Jul 17 13:20:08.037819 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.037794 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:08.039654 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.039637 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5j8xf" Jul 17 13:20:08.051891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.051877 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xlfpb" Jul 17 13:20:08.095394 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.095371 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:08.095490 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.095404 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:08.098532 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.098513 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 13:20:08.098644 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.098558 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 13:20:08.098644 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.098565 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5p8qj\"" Jul 17 13:20:08.120594 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.120560 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:08.150721 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.150656 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9j845" Jul 17 13:20:08.192303 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.192129 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:08.195329 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.195249 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq"] Jul 17 13:20:08.208309 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.208088 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:08.209342 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.209263 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq"] Jul 17 13:20:08.211551 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.211159 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-n52qh\"" Jul 17 13:20:08.211551 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.211398 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 17 13:20:08.256491 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.256465 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xlfpb"] Jul 17 13:20:08.259924 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.259880 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5j8xf"] Jul 17 13:20:08.260084 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:08.260058 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf927fd7f_9e9a_4af5_8fd7_76c057df969d.slice/crio-2256c71c74ce7bb28330249b1e5f187985752aec1be5fe32d109f70099f59293 WatchSource:0}: Error finding container 2256c71c74ce7bb28330249b1e5f187985752aec1be5fe32d109f70099f59293: Status 404 returned error can't find the container with id 2256c71c74ce7bb28330249b1e5f187985752aec1be5fe32d109f70099f59293 Jul 17 13:20:08.264368 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:08.263796 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3def5554_1674_4cfe_9da7_16f997510485.slice/crio-fe7bdcc5e71a58d2414ffa61c6cc11d66e2098277510ec8bafc17d0654435eb2 WatchSource:0}: Error finding container fe7bdcc5e71a58d2414ffa61c6cc11d66e2098277510ec8bafc17d0654435eb2: Status 404 returned error can't find the container with id fe7bdcc5e71a58d2414ffa61c6cc11d66e2098277510ec8bafc17d0654435eb2 Jul 17 13:20:08.266125 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.266102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/64e794d3-30e3-4447-9f4b-6516c3ed9af3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-n9qkq\" (UID: \"64e794d3-30e3-4447-9f4b-6516c3ed9af3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:08.274778 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.274753 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xlfpb" event={"ID":"f927fd7f-9e9a-4af5-8fd7-76c057df969d","Type":"ContainerStarted","Data":"2256c71c74ce7bb28330249b1e5f187985752aec1be5fe32d109f70099f59293"} Jul 17 13:20:08.275872 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.275852 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j8xf" event={"ID":"3def5554-1674-4cfe-9da7-16f997510485","Type":"ContainerStarted","Data":"fe7bdcc5e71a58d2414ffa61c6cc11d66e2098277510ec8bafc17d0654435eb2"} Jul 17 13:20:08.308732 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.308691 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9j845"] Jul 17 13:20:08.311349 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:08.311329 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65e5d8d_b568_4dea_a911_762096915096.slice/crio-f327e7df47839d7646959388a17587ca5b08e164fb65fe9b7e134de06e46585b WatchSource:0}: Error finding container f327e7df47839d7646959388a17587ca5b08e164fb65fe9b7e134de06e46585b: Status 404 returned error can't find the container with id f327e7df47839d7646959388a17587ca5b08e164fb65fe9b7e134de06e46585b Jul 17 13:20:08.367227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.367196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/64e794d3-30e3-4447-9f4b-6516c3ed9af3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-n9qkq\" (UID: \"64e794d3-30e3-4447-9f4b-6516c3ed9af3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:08.367339 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:08.367321 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 17 13:20:08.367394 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:08.367385 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e794d3-30e3-4447-9f4b-6516c3ed9af3-tls-certificates podName:64e794d3-30e3-4447-9f4b-6516c3ed9af3 nodeName:}" failed. No retries permitted until 2026-07-17 13:20:08.867365717 +0000 UTC m=+50.945479365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/64e794d3-30e3-4447-9f4b-6516c3ed9af3-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-n9qkq" (UID: "64e794d3-30e3-4447-9f4b-6516c3ed9af3") : secret "prometheus-operator-admission-webhook-tls" not found Jul 17 13:20:08.428785 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.428762 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:08.870395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.870358 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/64e794d3-30e3-4447-9f4b-6516c3ed9af3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-n9qkq\" (UID: \"64e794d3-30e3-4447-9f4b-6516c3ed9af3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:08.876607 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:08.876559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/64e794d3-30e3-4447-9f4b-6516c3ed9af3-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-n9qkq\" (UID: \"64e794d3-30e3-4447-9f4b-6516c3ed9af3\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:09.100815 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:09.100084 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:09.103076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:09.102872 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 13:20:09.103076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:09.102873 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7jn4x\"" Jul 17 13:20:09.103076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:09.102982 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 13:20:09.121025 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:09.120962 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:09.290233 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:09.289960 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9j845" event={"ID":"e65e5d8d-b568-4dea-a911-762096915096","Type":"ContainerStarted","Data":"f327e7df47839d7646959388a17587ca5b08e164fb65fe9b7e134de06e46585b"} Jul 17 13:20:09.337342 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:09.336198 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq"] Jul 17 13:20:09.341598 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:09.341560 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64e794d3_30e3_4447_9f4b_6516c3ed9af3.slice/crio-172fb58bd9e1956882e1333b96b2a0da6abe5d29ca8a9a0795d056c12b8b1222 WatchSource:0}: Error finding container 172fb58bd9e1956882e1333b96b2a0da6abe5d29ca8a9a0795d056c12b8b1222: Status 404 returned error can't find the container with id 172fb58bd9e1956882e1333b96b2a0da6abe5d29ca8a9a0795d056c12b8b1222 Jul 17 13:20:10.295989 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:10.295950 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" event={"ID":"64e794d3-30e3-4447-9f4b-6516c3ed9af3","Type":"ContainerStarted","Data":"172fb58bd9e1956882e1333b96b2a0da6abe5d29ca8a9a0795d056c12b8b1222"} Jul 17 13:20:11.163122 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:11.163075 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:11.216237 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:11.216189 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:11.645899 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:11.645851 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:11.750366 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:11.750319 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:12.125024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.124992 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dfcfd868c-7b46x"] Jul 17 13:20:12.132558 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.132532 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.137076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.136856 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 17 13:20:12.137076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.136874 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 17 13:20:12.137076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.136882 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 17 13:20:12.137076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.136893 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 17 13:20:12.137076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.136890 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 17 13:20:12.137076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.137017 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2xnbq\"" Jul 17 13:20:12.137076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.136995 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 17 13:20:12.137468 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.137437 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 17 13:20:12.138499 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.138475 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfcfd868c-7b46x"] Jul 17 13:20:12.201186 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.201148 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-oauth-config\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.201186 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.201188 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-config\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.201395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.201214 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-oauth-serving-cert\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.201395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.201292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-serving-cert\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.201395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.201326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-service-ca\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.201395 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.201359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7wt5\" (UniqueName: \"kubernetes.io/projected/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-kube-api-access-d7wt5\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.301890 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.301845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-oauth-config\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.301890 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.301895 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-config\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.302224 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.301953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-oauth-serving-cert\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.302224 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.301987 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-serving-cert\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.302224 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.302012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-service-ca\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.302956 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.302445 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7wt5\" (UniqueName: \"kubernetes.io/projected/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-kube-api-access-d7wt5\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.302956 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.302641 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-config\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.302956 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.302744 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-oauth-serving-cert\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.302956 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.302802 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-service-ca\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.305355 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.305312 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-oauth-config\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.305355 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.305349 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-serving-cert\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.311392 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.311373 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7wt5\" (UniqueName: \"kubernetes.io/projected/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-kube-api-access-d7wt5\") pod \"console-5dfcfd868c-7b46x\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:12.445414 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:12.445380 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:13.546146 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:13.546060 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:13.598534 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:13.598502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:14.491965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.486687 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dfcfd868c-7b46x"] Jul 17 13:20:14.529776 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.529750 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f9bbd547d-qrmjz"] Jul 17 13:20:14.532134 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.532118 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.546079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.545356 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 17 13:20:14.546406 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.546381 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f9bbd547d-qrmjz"] Jul 17 13:20:14.550402 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.550367 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:14.621218 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.621125 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-oauth-config\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.621218 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.621168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-serving-cert\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.621218 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.621201 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-oauth-serving-cert\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.621450 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.621252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-service-ca\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.621450 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.621274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-trusted-ca-bundle\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.621450 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.621306 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-config\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.621450 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.621363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47rtj\" (UniqueName: \"kubernetes.io/projected/c7386392-9f3b-4ea9-a1d4-d273e7319bee-kube-api-access-47rtj\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.641521 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.641482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:14.722144 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722111 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-service-ca\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.722144 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722148 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-trusted-ca-bundle\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.722360 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-config\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.722360 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47rtj\" (UniqueName: \"kubernetes.io/projected/c7386392-9f3b-4ea9-a1d4-d273e7319bee-kube-api-access-47rtj\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.722469 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-oauth-config\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.722469 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722421 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-serving-cert\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.722563 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-oauth-serving-cert\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.722916 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.722879 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-service-ca\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.723097 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.723052 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-config\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.723163 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.723103 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-oauth-serving-cert\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.723252 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.723234 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-trusted-ca-bundle\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.724824 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.724803 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-serving-cert\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.724824 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.724813 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-oauth-config\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.730446 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.730428 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47rtj\" (UniqueName: \"kubernetes.io/projected/c7386392-9f3b-4ea9-a1d4-d273e7319bee-kube-api-access-47rtj\") pod \"console-7f9bbd547d-qrmjz\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:14.847390 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:14.847345 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:15.020447 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.020413 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f9bbd547d-qrmjz"] Jul 17 13:20:15.026557 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:15.026515 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7386392_9f3b_4ea9_a1d4_d273e7319bee.slice/crio-b2c4f6b60b7466e5141183bef5836716a248dd98ee78b19c30719586207f1c1c WatchSource:0}: Error finding container b2c4f6b60b7466e5141183bef5836716a248dd98ee78b19c30719586207f1c1c: Status 404 returned error can't find the container with id b2c4f6b60b7466e5141183bef5836716a248dd98ee78b19c30719586207f1c1c Jul 17 13:20:15.313333 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.313294 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9j845" event={"ID":"e65e5d8d-b568-4dea-a911-762096915096","Type":"ContainerStarted","Data":"3e1da8bb49778d2cff25a928ae5155213e756c11995b621b41db3d85aea477b3"} Jul 17 13:20:15.313503 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.313356 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9j845" event={"ID":"e65e5d8d-b568-4dea-a911-762096915096","Type":"ContainerStarted","Data":"fc3b93f478c89f360edad890e6f6eadc2ea7cb00fc147fe9882746b8c0b16c38"} Jul 17 13:20:15.313503 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.313491 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9j845" Jul 17 13:20:15.315978 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.315945 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j8xf" event={"ID":"3def5554-1674-4cfe-9da7-16f997510485","Type":"ContainerStarted","Data":"0644d698444076ed84d672c8b42559cfb0d02bd5145a0a4a1608cf5d7c8f64c8"} Jul 17 13:20:15.318129 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.318103 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9bbd547d-qrmjz" event={"ID":"c7386392-9f3b-4ea9-a1d4-d273e7319bee","Type":"ContainerStarted","Data":"b2c4f6b60b7466e5141183bef5836716a248dd98ee78b19c30719586207f1c1c"} Jul 17 13:20:15.320203 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.320175 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfcfd868c-7b46x" event={"ID":"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82","Type":"ContainerStarted","Data":"91ff7bb13e31c476306881b161a7ec6a22444c8af57929d781795f8a55eae9e8"} Jul 17 13:20:15.322285 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.322054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" event={"ID":"64e794d3-30e3-4447-9f4b-6516c3ed9af3","Type":"ContainerStarted","Data":"fbcfea154b48b4f87a2233f31950b0070352e6e1f58f8dc43760ec0ef34212bd"} Jul 17 13:20:15.322285 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.322214 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:15.324151 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.324124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xlfpb" event={"ID":"f927fd7f-9e9a-4af5-8fd7-76c057df969d","Type":"ContainerStarted","Data":"571787972cbea2fce6fab97363f6517573bb6a850f8e3a75f383cd29c1ba13c9"} Jul 17 13:20:15.329047 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.329021 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" Jul 17 13:20:15.334710 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.334219 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9j845" podStartSLOduration=2.6315807810000003 podStartE2EDuration="8.334202719s" totalImagesPullingTime="5.702621938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:15.333760564 +0000 UTC m=+57.411874228" watchObservedRunningTime="2026-07-17 13:20:15.334202719 +0000 UTC m=+57.412316385" Jul 17 13:20:15.378124 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.377550 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xlfpb" podStartSLOduration=2.298759672 podStartE2EDuration="8.377533785s" totalImagesPullingTime="6.078774113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:15.376186836 +0000 UTC m=+57.454300505" watchObservedRunningTime="2026-07-17 13:20:15.377533785 +0000 UTC m=+57.455647441" Jul 17 13:20:15.378124 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.377839 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-n9qkq" podStartSLOduration=2.382120557 podStartE2EDuration="7.377833301s" totalImagesPullingTime="4.995712744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:15.355493009 +0000 UTC m=+57.433606674" watchObservedRunningTime="2026-07-17 13:20:15.377833301 +0000 UTC m=+57.455946968" Jul 17 13:20:15.468678 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.468639 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:15.570942 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:15.570835 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:16.245966 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.245925 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:16.282722 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.281690 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj"] Jul 17 13:20:16.310839 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.310738 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj"] Jul 17 13:20:16.311002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.310863 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.316458 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.316415 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 13:20:16.316683 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.316666 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 17 13:20:16.316884 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.316868 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rbs6t\"" Jul 17 13:20:16.317069 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.317052 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 17 13:20:16.317423 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.317406 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 13:20:16.317620 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.317602 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 13:20:16.339167 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.339140 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:16.439159 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.439126 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/216a0da8-fb64-403c-8291-54f620df8fa8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.439342 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.439207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwht9\" (UniqueName: \"kubernetes.io/projected/216a0da8-fb64-403c-8291-54f620df8fa8-kube-api-access-cwht9\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.439342 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.439286 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.439342 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.439312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.540667 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.540592 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/216a0da8-fb64-403c-8291-54f620df8fa8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.540667 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.540645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwht9\" (UniqueName: \"kubernetes.io/projected/216a0da8-fb64-403c-8291-54f620df8fa8-kube-api-access-cwht9\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.540881 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.540718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.540881 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.540744 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.541288 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:16.541262 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 17 13:20:16.541378 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:16.541345 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-tls podName:216a0da8-fb64-403c-8291-54f620df8fa8 nodeName:}" failed. No retries permitted until 2026-07-17 13:20:17.04132511 +0000 UTC m=+59.119438773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-jlqcj" (UID: "216a0da8-fb64-403c-8291-54f620df8fa8") : secret "prometheus-operator-tls" not found Jul 17 13:20:16.541697 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.541672 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/216a0da8-fb64-403c-8291-54f620df8fa8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.544598 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.544572 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.552963 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.552941 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwht9\" (UniqueName: \"kubernetes.io/projected/216a0da8-fb64-403c-8291-54f620df8fa8-kube-api-access-cwht9\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:16.622119 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:16.622084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:17.044273 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:17.044233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:17.046754 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:17.046729 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/216a0da8-fb64-403c-8291-54f620df8fa8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-jlqcj\" (UID: \"216a0da8-fb64-403c-8291-54f620df8fa8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:17.225785 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:17.225743 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" Jul 17 13:20:17.524459 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:17.524428 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:17.625231 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:17.625187 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:18.229926 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:18.229885 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s478r" Jul 17 13:20:18.355265 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:18.355232 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:18.411687 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:18.411639 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:19.402888 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:19.402810 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj"] Jul 17 13:20:19.405625 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:19.405595 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216a0da8_fb64_403c_8291_54f620df8fa8.slice/crio-60604b8cf04ffd3de347afb27132aafea0b03c75bd6b07daf134567f80c15908 WatchSource:0}: Error finding container 60604b8cf04ffd3de347afb27132aafea0b03c75bd6b07daf134567f80c15908: Status 404 returned error can't find the container with id 60604b8cf04ffd3de347afb27132aafea0b03c75bd6b07daf134567f80c15908 Jul 17 13:20:20.107961 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.107922 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:20.208326 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.208229 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:20.308126 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.308041 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:20.344973 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.344938 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j8xf" event={"ID":"3def5554-1674-4cfe-9da7-16f997510485","Type":"ContainerStarted","Data":"146058fb102efa17a4ac028b0dff9ce9e756fe15ed52b821a55b6b9b941cf888"} Jul 17 13:20:20.347079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.347048 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9bbd547d-qrmjz" event={"ID":"c7386392-9f3b-4ea9-a1d4-d273e7319bee","Type":"ContainerStarted","Data":"b22b4ae8f6a867bd0f17e8a68cb2d12bd1057ecd93758379594821fd21ac7953"} Jul 17 13:20:20.349310 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.349283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfcfd868c-7b46x" event={"ID":"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82","Type":"ContainerStarted","Data":"fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8"} Jul 17 13:20:20.350774 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.350737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" event={"ID":"216a0da8-fb64-403c-8291-54f620df8fa8","Type":"ContainerStarted","Data":"60604b8cf04ffd3de347afb27132aafea0b03c75bd6b07daf134567f80c15908"} Jul 17 13:20:20.364793 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.364704 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f9bbd547d-qrmjz" podStartSLOduration=2.114374608 podStartE2EDuration="6.364686713s" totalImagesPullingTime="4.250312105s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:20.364161006 +0000 UTC m=+62.442274677" watchObservedRunningTime="2026-07-17 13:20:20.364686713 +0000 UTC m=+62.442800377" Jul 17 13:20:20.380390 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:20.380347 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dfcfd868c-7b46x" podStartSLOduration=3.607250188 podStartE2EDuration="8.380334225s" totalImagesPullingTime="4.773084037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:20.379783867 +0000 UTC m=+62.457897532" watchObservedRunningTime="2026-07-17 13:20:20.380334225 +0000 UTC m=+62.458447888" Jul 17 13:20:21.299831 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:21.299737 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:21.355628 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:21.355588 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" event={"ID":"216a0da8-fb64-403c-8291-54f620df8fa8","Type":"ContainerStarted","Data":"3b70333b9bb3dce8413ce2711e84ee3bdcdbe16d6823397d2ea876731b514463"} Jul 17 13:20:21.355793 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:21.355655 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" event={"ID":"216a0da8-fb64-403c-8291-54f620df8fa8","Type":"ContainerStarted","Data":"cc3030d28d7755f3acfe94d7f8a2f64c638f10a23d1483998be4795d82ab0eba"} Jul 17 13:20:21.372420 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:21.372370 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-jlqcj" podStartSLOduration=3.804829494 podStartE2EDuration="5.3723546s" totalImagesPullingTime="1.567525106s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:21.371653151 +0000 UTC m=+63.449766815" watchObservedRunningTime="2026-07-17 13:20:21.3723546 +0000 UTC m=+63.450468266" Jul 17 13:20:21.376642 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:21.376609 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:22.446556 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:22.446517 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:22.447012 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:22.446612 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:22.453026 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:22.453002 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:22.509361 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:22.509322 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:22.626544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:22.626505 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:23.368202 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.368179 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:23.386402 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.386369 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:23.606023 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.605989 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:23.631633 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.631566 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q"] Jul 17 13:20:23.653114 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.653087 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q"] Jul 17 13:20:23.653226 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.653146 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-s9j7s"] Jul 17 13:20:23.653335 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.653315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.656053 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.656033 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 17 13:20:23.656254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.656232 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-g99qx\"" Jul 17 13:20:23.656712 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.656698 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 17 13:20:23.675479 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.675460 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8c9sg"] Jul 17 13:20:23.675580 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.675569 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.677972 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.677941 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 17 13:20:23.678133 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.678113 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-wxnwf\"" Jul 17 13:20:23.678216 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.678131 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 17 13:20:23.678339 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.678276 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 17 13:20:23.679432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.679410 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:23.697193 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.697170 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-s9j7s"] Jul 17 13:20:23.697283 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.697272 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.699731 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.699711 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 13:20:23.699814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.699792 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 13:20:23.699860 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.699819 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5jhwq\"" Jul 17 13:20:23.700119 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.700099 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 13:20:23.700257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.700230 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.700366 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.700312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2gr7\" (UniqueName: \"kubernetes.io/projected/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-kube-api-access-d2gr7\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.700366 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.700358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.700461 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.700443 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.746135 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.746101 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:23.801533 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801512 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.801649 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801541 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.801649 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd3654e-332f-40c0-aed6-026887beae62-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.801649 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801589 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gr7\" (UniqueName: \"kubernetes.io/projected/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-kube-api-access-d2gr7\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.801649 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzr6t\" (UniqueName: \"kubernetes.io/projected/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-kube-api-access-bzr6t\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.801792 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:23.801655 2560 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 17 13:20:23.801792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801672 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.801792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-accelerators-collector-config\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.801792 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:23.801747 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-tls podName:9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2 nodeName:}" failed. No retries permitted until 2026-07-17 13:20:24.301728298 +0000 UTC m=+66.379841955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-vf52q" (UID: "9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2") : secret "openshift-state-metrics-tls" not found Jul 17 13:20:23.801792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801764 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.801792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xcdk\" (UniqueName: \"kubernetes.io/projected/bbd3654e-332f-40c0-aed6-026887beae62-kube-api-access-4xcdk\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-root\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801901 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-tls\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801947 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bbd3654e-332f-40c0-aed6-026887beae62-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.801988 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-sys\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.802024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.802006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-wtmp\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.802311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.802029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-metrics-client-ca\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.802311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.802064 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-textfile\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.802566 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.802550 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.804020 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.804004 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.810074 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.810051 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2gr7\" (UniqueName: \"kubernetes.io/projected/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-kube-api-access-d2gr7\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:23.845465 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.845436 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:23.903034 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.902970 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.903034 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903004 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd3654e-332f-40c0-aed6-026887beae62-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.903197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzr6t\" (UniqueName: \"kubernetes.io/projected/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-kube-api-access-bzr6t\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903070 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-accelerators-collector-config\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903126 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.903197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903153 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xcdk\" (UniqueName: \"kubernetes.io/projected/bbd3654e-332f-40c0-aed6-026887beae62-kube-api-access-4xcdk\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.903197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903192 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-root\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-tls\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:23.903268 2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bbd3654e-332f-40c0-aed6-026887beae62-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903318 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-sys\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:23.903343 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-tls podName:bbd3654e-332f-40c0-aed6-026887beae62 nodeName:}" failed. No retries permitted until 2026-07-17 13:20:24.403323048 +0000 UTC m=+66.481436691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-s9j7s" (UID: "bbd3654e-332f-40c0-aed6-026887beae62") : secret "kube-state-metrics-tls" not found Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903371 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-sys\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-wtmp\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903430 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-metrics-client-ca\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903497 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-textfile\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903517 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-wtmp\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903754 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd3654e-332f-40c0-aed6-026887beae62-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.903960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903818 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-root\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.903960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903881 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/bbd3654e-332f-40c0-aed6-026887beae62-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.904142 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.903961 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-accelerators-collector-config\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.904142 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.904128 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-textfile\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.904319 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.904299 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-metrics-client-ca\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.904560 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.904545 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.905490 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.905443 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.905898 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.905880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-node-exporter-tls\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.905963 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.905899 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.913572 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.913553 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzr6t\" (UniqueName: \"kubernetes.io/projected/dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31-kube-api-access-bzr6t\") pod \"node-exporter-8c9sg\" (UID: \"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31\") " pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:23.913861 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.913843 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xcdk\" (UniqueName: \"kubernetes.io/projected/bbd3654e-332f-40c0-aed6-026887beae62-kube-api-access-4xcdk\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:23.949634 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:23.949609 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.005388 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.005362 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8c9sg" Jul 17 13:20:24.014692 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:24.014670 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbd3e54f_3fa8_4e1d_8a8f_b815f75d8b31.slice/crio-1ffc4d56b71f4eb246c506cc040eada78d0ebce315aaaf963128db75c1c682bd WatchSource:0}: Error finding container 1ffc4d56b71f4eb246c506cc040eada78d0ebce315aaaf963128db75c1c682bd: Status 404 returned error can't find the container with id 1ffc4d56b71f4eb246c506cc040eada78d0ebce315aaaf963128db75c1c682bd Jul 17 13:20:24.046706 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.046677 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.105066 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.105039 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:24.107535 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.107518 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 13:20:24.117746 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.117730 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1992cdcc-16ba-4ebe-a0a8-b382deed8fcb-metrics-certs\") pod \"network-metrics-daemon-pfz9l\" (UID: \"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb\") " pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:24.147486 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.147461 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.206067 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.206037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:24.208602 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.208585 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 13:20:24.218601 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.218576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b91e071d-432f-4a3d-8f62-f18403981b2d-original-pull-secret\") pod \"global-pull-secret-syncer-9b6c9\" (UID: \"b91e071d-432f-4a3d-8f62-f18403981b2d\") " pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:24.247581 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.247556 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.305941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.305905 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9b6c9" Jul 17 13:20:24.307139 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.307124 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:24.307193 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.307152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:24.309460 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.309440 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-vf52q\" (UID: \"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:24.309893 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.309875 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 13:20:24.314258 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.314237 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5p8qj\"" Jul 17 13:20:24.319941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.319906 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 13:20:24.322934 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.322922 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pfz9l" Jul 17 13:20:24.330220 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.330199 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l9pd\" (UniqueName: \"kubernetes.io/projected/2441c8b2-c301-47f9-96f2-eb0049a7dda3-kube-api-access-7l9pd\") pod \"network-check-target-8nhwn\" (UID: \"2441c8b2-c301-47f9-96f2-eb0049a7dda3\") " pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:24.371953 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.371304 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5j8xf" event={"ID":"3def5554-1674-4cfe-9da7-16f997510485","Type":"ContainerStarted","Data":"df65bbb49f5d1cc7502efb21ed0e17b5e96ef9ddb4fd03b0f65af03b5dc8d2be"} Jul 17 13:20:24.373443 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.373411 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c9sg" event={"ID":"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31","Type":"ContainerStarted","Data":"1ffc4d56b71f4eb246c506cc040eada78d0ebce315aaaf963128db75c1c682bd"} Jul 17 13:20:24.390602 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.390555 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5j8xf" podStartSLOduration=2.364418875 podStartE2EDuration="17.390541222s" totalImagesPullingTime="15.026122347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:20:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:24.389243071 +0000 UTC m=+66.467356737" watchObservedRunningTime="2026-07-17 13:20:24.390541222 +0000 UTC m=+66.468654886" Jul 17 13:20:24.408495 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.408470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:24.410643 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.410622 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/bbd3654e-332f-40c0-aed6-026887beae62-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-s9j7s\" (UID: \"bbd3654e-332f-40c0-aed6-026887beae62\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:24.416485 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.416075 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7jn4x\"" Jul 17 13:20:24.424238 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.424212 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:24.440969 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.440881 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9b6c9"] Jul 17 13:20:24.444294 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:24.444263 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb91e071d_432f_4a3d_8f62_f18403981b2d.slice/crio-4b811031a0bb13cd4ffdc21c192f5739a900a163a1dac78d4f67c798c0f31314 WatchSource:0}: Error finding container 4b811031a0bb13cd4ffdc21c192f5739a900a163a1dac78d4f67c798c0f31314: Status 404 returned error can't find the container with id 4b811031a0bb13cd4ffdc21c192f5739a900a163a1dac78d4f67c798c0f31314 Jul 17 13:20:24.459462 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.459438 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pfz9l"] Jul 17 13:20:24.461923 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:24.461891 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1992cdcc_16ba_4ebe_a0a8_b382deed8fcb.slice/crio-e61a0f5f8c1dea4a04388d81a9574c34067cb129f9188666945541c28e0c1ce5 WatchSource:0}: Error finding container e61a0f5f8c1dea4a04388d81a9574c34067cb129f9188666945541c28e0c1ce5: Status 404 returned error can't find the container with id e61a0f5f8c1dea4a04388d81a9574c34067cb129f9188666945541c28e0c1ce5 Jul 17 13:20:24.551208 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.551150 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8nhwn"] Jul 17 13:20:24.553320 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:24.553279 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2441c8b2_c301_47f9_96f2_eb0049a7dda3.slice/crio-6fafacca3e81f611b06a83774e7a7135dea16267f3f09c3a69d6b72143cf702f WatchSource:0}: Error finding container 6fafacca3e81f611b06a83774e7a7135dea16267f3f09c3a69d6b72143cf702f: Status 404 returned error can't find the container with id 6fafacca3e81f611b06a83774e7a7135dea16267f3f09c3a69d6b72143cf702f Jul 17 13:20:24.563286 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.563265 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" Jul 17 13:20:24.584504 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.584484 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" Jul 17 13:20:24.624058 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.623984 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.702197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.701658 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q"] Jul 17 13:20:24.705361 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.705213 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.719155 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.719127 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-s9j7s"] Jul 17 13:20:24.721813 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:24.721786 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd3654e_332f_40c0_aed6_026887beae62.slice/crio-63f7c683a7a5907702e7722d0946ac75efabaaeeb8ce00ec7dd20011a2a2603f WatchSource:0}: Error finding container 63f7c683a7a5907702e7722d0946ac75efabaaeeb8ce00ec7dd20011a2a2603f: Status 404 returned error can't find the container with id 63f7c683a7a5907702e7722d0946ac75efabaaeeb8ce00ec7dd20011a2a2603f Jul 17 13:20:24.736369 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.736345 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:20:24.739513 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.739494 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.743744 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.743722 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hg5f6\"" Jul 17 13:20:24.743841 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.743825 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 17 13:20:24.744038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744021 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 17 13:20:24.744186 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744170 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 17 13:20:24.744309 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 17 13:20:24.744359 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744025 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 17 13:20:24.744411 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744397 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 17 13:20:24.744460 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744406 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 17 13:20:24.744460 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744420 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 17 13:20:24.744460 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.744432 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 17 13:20:24.758822 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.758395 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:20:24.782274 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.782248 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.811761 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.811738 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-web-config\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.811877 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.811811 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.811953 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.811887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812006 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.811960 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812006 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.811995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpvxj\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-kube-api-access-vpvxj\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812133 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812182 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812153 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-config-out\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812182 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812246 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812246 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812235 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812319 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812319 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812293 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-config-volume\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.812401 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.812382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.845513 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.845481 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:24.847769 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.847702 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:24.848599 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.847928 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:24.854889 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.854676 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:24.913342 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913512 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913351 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-config-out\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913512 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913377 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913512 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913607 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913638 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913757 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913671 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-config-volume\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913757 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913852 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913760 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-web-config\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913852 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913969 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913858 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913969 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913884 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.913969 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.913928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpvxj\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-kube-api-access-vpvxj\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.915645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.915615 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.917024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.916224 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-config-out\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.917024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.916367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.917024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.916433 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.917024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.916669 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.917024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.916984 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.918082 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.918035 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-tls-assets\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.918311 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.918287 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.918425 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.918378 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.918897 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.918876 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-web-config\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.919012 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.918966 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.919263 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.919245 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-config-volume\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:24.922697 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:24.922679 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpvxj\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-kube-api-access-vpvxj\") pod \"alertmanager-main-0\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:25.064712 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.064591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:20:25.235376 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.235212 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:20:25.311376 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.311329 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:25.330115 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.330044 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9j845" Jul 17 13:20:25.379363 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.379283 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nhwn" event={"ID":"2441c8b2-c301-47f9-96f2-eb0049a7dda3","Type":"ContainerStarted","Data":"6fafacca3e81f611b06a83774e7a7135dea16267f3f09c3a69d6b72143cf702f"} Jul 17 13:20:25.385841 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.385708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" event={"ID":"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2","Type":"ContainerStarted","Data":"4dc38e12fff0449bc09dadde25a507e0b61fb4b6beca1811b2a1ab678cec3767"} Jul 17 13:20:25.385841 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.385773 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" event={"ID":"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2","Type":"ContainerStarted","Data":"586bdb07aaf84aaf214626ebcc6131538b604044e0d548f88f13cc7ba755f6af"} Jul 17 13:20:25.385841 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.385805 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" event={"ID":"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2","Type":"ContainerStarted","Data":"4791adbba1dbed006afac506335dbd7ced7441967330c7462f97ab0bd778a56b"} Jul 17 13:20:25.387654 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.387624 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" event={"ID":"bbd3654e-332f-40c0-aed6-026887beae62","Type":"ContainerStarted","Data":"63f7c683a7a5907702e7722d0946ac75efabaaeeb8ce00ec7dd20011a2a2603f"} Jul 17 13:20:25.389487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.389417 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfz9l" event={"ID":"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb","Type":"ContainerStarted","Data":"e61a0f5f8c1dea4a04388d81a9574c34067cb129f9188666945541c28e0c1ce5"} Jul 17 13:20:25.392336 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.392308 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9b6c9" event={"ID":"b91e071d-432f-4a3d-8f62-f18403981b2d","Type":"ContainerStarted","Data":"4b811031a0bb13cd4ffdc21c192f5739a900a163a1dac78d4f67c798c0f31314"} Jul 17 13:20:25.398238 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.398031 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:20:25.421258 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.421177 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:25.460429 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.460339 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfcfd868c-7b46x"] Jul 17 13:20:25.631944 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.631850 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5c997d6f94-bf7np"] Jul 17 13:20:25.635074 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.635054 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.638484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.638247 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 17 13:20:25.638484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.638247 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4uetabau4s7hl\"" Jul 17 13:20:25.638484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.638394 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 17 13:20:25.638686 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.638559 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 17 13:20:25.638686 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.638576 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-xcbfj\"" Jul 17 13:20:25.638783 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.638725 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 17 13:20:25.638783 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.638758 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 17 13:20:25.649892 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.649782 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c997d6f94-bf7np"] Jul 17 13:20:25.720676 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.720643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.721005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.720711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.721005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.720735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-grpc-tls\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.721005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.720766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e110e86d-d307-469c-8ce9-1c369156ac8d-metrics-client-ca\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.721005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.720822 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.721005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.720847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.721005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.720971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-tls\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.721606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.721018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjdlz\" (UniqueName: \"kubernetes.io/projected/e110e86d-d307-469c-8ce9-1c369156ac8d-kube-api-access-fjdlz\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-grpc-tls\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822486 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e110e86d-d307-469c-8ce9-1c369156ac8d-metrics-client-ca\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822590 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822635 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-tls\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjdlz\" (UniqueName: \"kubernetes.io/projected/e110e86d-d307-469c-8ce9-1c369156ac8d-kube-api-access-fjdlz\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.823613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.822692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.824795 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.824164 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e110e86d-d307-469c-8ce9-1c369156ac8d-metrics-client-ca\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.828666 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.828631 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-tls\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.829502 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.829426 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.830116 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.829799 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.830954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.830928 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.831088 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.831040 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.831666 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.831644 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e110e86d-d307-469c-8ce9-1c369156ac8d-secret-grpc-tls\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.833499 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.833458 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjdlz\" (UniqueName: \"kubernetes.io/projected/e110e86d-d307-469c-8ce9-1c369156ac8d-kube-api-access-fjdlz\") pod \"thanos-querier-5c997d6f94-bf7np\" (UID: \"e110e86d-d307-469c-8ce9-1c369156ac8d\") " pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:25.953122 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:25.952327 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:26.161017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:26.160857 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5c997d6f94-bf7np"] Jul 17 13:20:26.400773 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:26.400737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerStarted","Data":"b3e8248f917647120ef06fbf46b8b059d695774c9dc6004128d8c66cb6239987"} Jul 17 13:20:26.402990 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:26.402832 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c9sg" event={"ID":"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31","Type":"ContainerStarted","Data":"26d951d1a67ae01a8b688c1bc9613fc28738b97d766dd95e87a63a1abb98fe1e"} Jul 17 13:20:26.406677 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:26.406648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfz9l" event={"ID":"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb","Type":"ContainerStarted","Data":"8b0cf0320c78f229866125730029a913a632d3fb9f89132de17965a5cf4a446e"} Jul 17 13:20:26.448885 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:26.448438 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode110e86d_d307_469c_8ce9_1c369156ac8d.slice/crio-a50bfd6ca003117d02430ed165d5dcb6bc4bc8eb1047faa683f95b6f6edf0a09 WatchSource:0}: Error finding container a50bfd6ca003117d02430ed165d5dcb6bc4bc8eb1047faa683f95b6f6edf0a09: Status 404 returned error can't find the container with id a50bfd6ca003117d02430ed165d5dcb6bc4bc8eb1047faa683f95b6f6edf0a09 Jul 17 13:20:26.612729 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:26.612688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:26.711933 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:26.711750 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:26.812052 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:26.812020 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:27.015567 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.015465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:27.404628 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.404506 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:27.412990 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.412795 2560 generic.go:356] "Generic (PLEG): container finished" podID="dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31" containerID="26d951d1a67ae01a8b688c1bc9613fc28738b97d766dd95e87a63a1abb98fe1e" exitCode=0 Jul 17 13:20:27.412990 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.412900 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c9sg" event={"ID":"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31","Type":"ContainerDied","Data":"26d951d1a67ae01a8b688c1bc9613fc28738b97d766dd95e87a63a1abb98fe1e"} Jul 17 13:20:27.425375 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.425287 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pfz9l" event={"ID":"1992cdcc-16ba-4ebe-a0a8-b382deed8fcb","Type":"ContainerStarted","Data":"38017606bcd75f809082aea4716534f825fa181cfd6f571bcefc1377ea387d85"} Jul 17 13:20:27.433859 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.433664 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" event={"ID":"e110e86d-d307-469c-8ce9-1c369156ac8d","Type":"ContainerStarted","Data":"a50bfd6ca003117d02430ed165d5dcb6bc4bc8eb1047faa683f95b6f6edf0a09"} Jul 17 13:20:27.449591 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.449540 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pfz9l" podStartSLOduration=66.99269227 podStartE2EDuration="1m8.449520754s" totalImagesPullingTime="1.456828484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:27.448712826 +0000 UTC m=+69.526826493" watchObservedRunningTime="2026-07-17 13:20:27.449520754 +0000 UTC m=+69.527634419" Jul 17 13:20:27.470801 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.470749 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:27.571895 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.571791 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:27.666547 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.666139 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:27.768039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.767864 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:27.821765 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:27.821680 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:28.025160 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.025123 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-76599cfbd6-w7cnp"] Jul 17 13:20:28.032289 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.032262 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.035389 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.035358 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76599cfbd6-w7cnp"] Jul 17 13:20:28.036278 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.035980 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 17 13:20:28.036278 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.036119 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 17 13:20:28.036278 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.035980 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 17 13:20:28.036479 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.036404 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 17 13:20:28.037357 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.037129 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8k1dl1r1tg1p9\"" Jul 17 13:20:28.037781 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.037744 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-46khn\"" Jul 17 13:20:28.145615 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.145579 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/20685541-c9ad-4c7c-bc06-21371aff740f-metrics-server-audit-profiles\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.145773 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.145652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/20685541-c9ad-4c7c-bc06-21371aff740f-audit-log\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.145879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.145859 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-secret-metrics-server-tls\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.145953 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.145890 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-secret-metrics-server-client-certs\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.145953 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.145946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njvnj\" (UniqueName: \"kubernetes.io/projected/20685541-c9ad-4c7c-bc06-21371aff740f-kube-api-access-njvnj\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.146060 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.145971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20685541-c9ad-4c7c-bc06-21371aff740f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.146060 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.145989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-client-ca-bundle\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.246873 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.246829 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-secret-metrics-server-tls\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.247077 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.246882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-secret-metrics-server-client-certs\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.247077 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.246933 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njvnj\" (UniqueName: \"kubernetes.io/projected/20685541-c9ad-4c7c-bc06-21371aff740f-kube-api-access-njvnj\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.247077 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.247011 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20685541-c9ad-4c7c-bc06-21371aff740f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.247077 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.247043 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-client-ca-bundle\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.247278 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.247104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/20685541-c9ad-4c7c-bc06-21371aff740f-metrics-server-audit-profiles\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.247278 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.247162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/20685541-c9ad-4c7c-bc06-21371aff740f-audit-log\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.247652 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.247626 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/20685541-c9ad-4c7c-bc06-21371aff740f-audit-log\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.248097 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.248026 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20685541-c9ad-4c7c-bc06-21371aff740f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.248426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.248397 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/20685541-c9ad-4c7c-bc06-21371aff740f-metrics-server-audit-profiles\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.249831 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.249810 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-secret-metrics-server-tls\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.250027 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.250005 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-client-ca-bundle\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.250097 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.250017 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/20685541-c9ad-4c7c-bc06-21371aff740f-secret-metrics-server-client-certs\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.255724 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.255694 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njvnj\" (UniqueName: \"kubernetes.io/projected/20685541-c9ad-4c7c-bc06-21371aff740f-kube-api-access-njvnj\") pod \"metrics-server-76599cfbd6-w7cnp\" (UID: \"20685541-c9ad-4c7c-bc06-21371aff740f\") " pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.348235 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.348157 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:28.411724 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.411694 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb"] Jul 17 13:20:28.412286 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.412254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:28.415098 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.415078 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:28.417699 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.417678 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 17 13:20:28.417847 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.417791 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-q4v4m\"" Jul 17 13:20:28.422936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.422855 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb"] Jul 17 13:20:28.441689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.441183 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" event={"ID":"bbd3654e-332f-40c0-aed6-026887beae62","Type":"ContainerStarted","Data":"0574477ee0b05b2f807c646470182692ae3dea9b05f18ebd2b120c9113652aa3"} Jul 17 13:20:28.441689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.441259 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" event={"ID":"bbd3654e-332f-40c0-aed6-026887beae62","Type":"ContainerStarted","Data":"e13d5f7cc39b57a63aaf2cd5eba30ba91b00c190b2b76b89bb3120eb89f05972"} Jul 17 13:20:28.441689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.441286 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" event={"ID":"bbd3654e-332f-40c0-aed6-026887beae62","Type":"ContainerStarted","Data":"03ad8ea88298baa21d7076a93edd402a89f9603f6da34d002fb999d2016af6f3"} Jul 17 13:20:28.443161 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.443133 2560 generic.go:356] "Generic (PLEG): container finished" podID="a986e594-7bc6-4a3c-871b-c821e4206544" containerID="e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed" exitCode=0 Jul 17 13:20:28.443330 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.443287 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed"} Jul 17 13:20:28.446539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.446514 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c9sg" event={"ID":"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31","Type":"ContainerStarted","Data":"8c706658dc07eaa33a38371d3834d2a031b361beb02529126636f5f46b2e94a5"} Jul 17 13:20:28.446638 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.446564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8c9sg" event={"ID":"dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31","Type":"ContainerStarted","Data":"4b7d3cfc22ab3f15df941164410fef19ba2a97720e54e78e67151de9e5aca1bf"} Jul 17 13:20:28.449838 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.449811 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" event={"ID":"9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2","Type":"ContainerStarted","Data":"5ba25ebfafaa4f150f3150d0a32874c01cd1ba25f0942979dab6cf00eb9fb4aa"} Jul 17 13:20:28.464535 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.464492 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-s9j7s" podStartSLOduration=2.852930548 podStartE2EDuration="5.464473488s" totalImagesPullingTime="2.61154294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:28.459246103 +0000 UTC m=+70.537359781" watchObservedRunningTime="2026-07-17 13:20:28.464473488 +0000 UTC m=+70.542587156" Jul 17 13:20:28.490829 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.490796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:28.504376 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.504325 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8c9sg" podStartSLOduration=3.460662435 podStartE2EDuration="5.504304642s" totalImagesPullingTime="1.900500818s" totalInitContainerRuntime="143.141389ms" isStatefulPod=true podCreationTimestamp="2026-07-17 13:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:28.502471295 +0000 UTC m=+70.580584982" watchObservedRunningTime="2026-07-17 13:20:28.504304642 +0000 UTC m=+70.582418307" Jul 17 13:20:28.519336 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.519289 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-vf52q" podStartSLOduration=3.022363833 podStartE2EDuration="5.51927211s" totalImagesPullingTime="2.496908277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:28.518367483 +0000 UTC m=+70.596481148" watchObservedRunningTime="2026-07-17 13:20:28.51927211 +0000 UTC m=+70.597385776" Jul 17 13:20:28.550248 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.550209 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/be85d966-ec82-4885-932f-143a63405dbd-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pr6wb\" (UID: \"be85d966-ec82-4885-932f-143a63405dbd\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:28.550690 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.550668 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jps4\" (UniqueName: \"kubernetes.io/projected/be85d966-ec82-4885-932f-143a63405dbd-kube-api-access-6jps4\") pod \"monitoring-plugin-7b7b7bc8bf-pr6wb\" (UID: \"be85d966-ec82-4885-932f-143a63405dbd\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:28.651937 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.651655 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/be85d966-ec82-4885-932f-143a63405dbd-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pr6wb\" (UID: \"be85d966-ec82-4885-932f-143a63405dbd\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:28.651937 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.651731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jps4\" (UniqueName: \"kubernetes.io/projected/be85d966-ec82-4885-932f-143a63405dbd-kube-api-access-6jps4\") pod \"monitoring-plugin-7b7b7bc8bf-pr6wb\" (UID: \"be85d966-ec82-4885-932f-143a63405dbd\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:28.652205 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:28.652181 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 17 13:20:28.652404 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:28.652254 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be85d966-ec82-4885-932f-143a63405dbd-monitoring-plugin-cert podName:be85d966-ec82-4885-932f-143a63405dbd nodeName:}" failed. No retries permitted until 2026-07-17 13:20:29.152231583 +0000 UTC m=+71.230345225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/be85d966-ec82-4885-932f-143a63405dbd-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-pr6wb" (UID: "be85d966-ec82-4885-932f-143a63405dbd") : secret "monitoring-plugin-cert" not found Jul 17 13:20:28.665804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.665772 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jps4\" (UniqueName: \"kubernetes.io/projected/be85d966-ec82-4885-932f-143a63405dbd-kube-api-access-6jps4\") pod \"monitoring-plugin-7b7b7bc8bf-pr6wb\" (UID: \"be85d966-ec82-4885-932f-143a63405dbd\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:28.697581 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.697525 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:28.837680 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.837644 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-555969c789-bscs4"] Jul 17 13:20:28.841572 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.841544 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.844467 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.844442 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c6v2l\"" Jul 17 13:20:28.844598 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.844487 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 17 13:20:28.844675 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.844598 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 17 13:20:28.844839 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.844823 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 17 13:20:28.844939 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.844876 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 17 13:20:28.845248 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.845230 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 17 13:20:28.850021 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.850000 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 17 13:20:28.855144 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.855123 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-555969c789-bscs4"] Jul 17 13:20:28.953842 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.953810 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-metrics-client-ca\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.954027 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.953853 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.954027 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.953881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htmmm\" (UniqueName: \"kubernetes.io/projected/6a664b5a-72aa-4fc3-8738-93d0a51744cc-kube-api-access-htmmm\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.954027 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.953936 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-telemeter-client-tls\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.954027 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.953973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.954027 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.953994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-federate-client-tls\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.954248 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.954098 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-serving-certs-ca-bundle\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:28.954248 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:28.954154 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-secret-telemeter-client\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.054811 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.054780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-metrics-client-ca\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055013 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.054821 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055013 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.054853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htmmm\" (UniqueName: \"kubernetes.io/projected/6a664b5a-72aa-4fc3-8738-93d0a51744cc-kube-api-access-htmmm\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055013 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.054893 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-telemeter-client-tls\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055013 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.054958 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055013 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.054988 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-federate-client-tls\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055274 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.055028 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-serving-certs-ca-bundle\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055274 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.055082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-secret-telemeter-client\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.055557 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.055525 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-metrics-client-ca\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.056239 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.056211 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-telemeter-trusted-ca-bundle\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.056603 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.056553 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a664b5a-72aa-4fc3-8738-93d0a51744cc-serving-certs-ca-bundle\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.057894 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.057869 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-secret-telemeter-client\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.058001 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.057984 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-federate-client-tls\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.058077 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.058057 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-telemeter-client-tls\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.058332 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.058313 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6a664b5a-72aa-4fc3-8738-93d0a51744cc-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.062721 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.062692 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htmmm\" (UniqueName: \"kubernetes.io/projected/6a664b5a-72aa-4fc3-8738-93d0a51744cc-kube-api-access-htmmm\") pod \"telemeter-client-555969c789-bscs4\" (UID: \"6a664b5a-72aa-4fc3-8738-93d0a51744cc\") " pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.156346 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.156314 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-555969c789-bscs4" Jul 17 13:20:29.156593 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.156561 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/be85d966-ec82-4885-932f-143a63405dbd-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pr6wb\" (UID: \"be85d966-ec82-4885-932f-143a63405dbd\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:29.159726 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.159385 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/be85d966-ec82-4885-932f-143a63405dbd-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-pr6wb\" (UID: \"be85d966-ec82-4885-932f-143a63405dbd\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:29.329103 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.329019 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:29.766930 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.766878 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-864cd47f86-fd84k"] Jul 17 13:20:29.770461 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.770432 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.779078 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.779047 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:29.780743 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.780452 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864cd47f86-fd84k"] Jul 17 13:20:29.854505 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.854467 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:29.864437 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.864410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-oauth-config\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.864580 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.864494 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-trusted-ca-bundle\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.864580 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.864528 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-service-ca\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.864580 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.864557 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-serving-cert\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.864750 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.864631 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-oauth-serving-cert\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.864750 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.864670 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6vj2\" (UniqueName: \"kubernetes.io/projected/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-kube-api-access-d6vj2\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.864750 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.864742 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-config\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.914017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.913985 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:20:29.918629 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.918493 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:29.921941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.921238 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 17 13:20:29.921941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.921377 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gzqtq\"" Jul 17 13:20:29.921941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.921534 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 17 13:20:29.921941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.921671 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 17 13:20:29.921941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.921714 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 17 13:20:29.921941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.921845 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 17 13:20:29.922298 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.922146 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-45dck21tsj2ek\"" Jul 17 13:20:29.922298 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.922188 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 17 13:20:29.924814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.922952 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 17 13:20:29.924814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.922995 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 17 13:20:29.924814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.923195 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 17 13:20:29.924814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.923268 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 17 13:20:29.924814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.923323 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 17 13:20:29.925949 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.925290 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 17 13:20:29.927695 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.927671 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 17 13:20:29.930231 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.930210 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 17 13:20:29.934530 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.934505 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:20:29.936546 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.936520 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:29.966514 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.965329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-oauth-serving-cert\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.966692 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.966537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6vj2\" (UniqueName: \"kubernetes.io/projected/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-kube-api-access-d6vj2\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.966692 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.966605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-config\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.966692 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.966634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-oauth-config\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.966838 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.966722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-trusted-ca-bundle\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.966838 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.966758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-service-ca\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.966838 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.966786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-serving-cert\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.967691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.967350 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-oauth-serving-cert\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.968795 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.968069 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-trusted-ca-bundle\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.968795 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.968635 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-config\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.968795 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.968734 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-service-ca\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.970613 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.970586 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-serving-cert\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.971951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.971903 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-oauth-config\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:29.975724 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:29.975570 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6vj2\" (UniqueName: \"kubernetes.io/projected/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-kube-api-access-d6vj2\") pod \"console-864cd47f86-fd84k\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:30.068270 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068187 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-web-config\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068270 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068241 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068478 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068298 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config-out\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068478 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068478 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068354 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068478 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068672 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068672 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068597 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068672 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068817 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068817 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068705 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068817 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068736 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.068817 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068769 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdkf8\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-kube-api-access-pdkf8\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.069035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.069035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.069035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068903 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.069035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.068981 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.069035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.069006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.069035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.069033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.069331 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.069068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.084205 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.084141 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:30.170122 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170272 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-web-config\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170272 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170272 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config-out\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170272 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170277 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.170501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.170376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171325 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171051 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171325 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171325 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171308 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171325 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171318 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171353 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171403 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171429 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdkf8\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-kube-api-access-pdkf8\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171539 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.171612 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.172018 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171616 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.172018 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171640 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.172018 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.171775 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.172018 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.172005 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.172633 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.172606 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.174977 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.174666 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.175837 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.175812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.176892 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.176068 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.176892 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.176679 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.177067 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.176975 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-web-config\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.177699 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.177403 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.178239 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.178175 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.178239 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.178179 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config-out\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.178239 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.178225 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.178440 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.178417 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.178713 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.178696 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.180301 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.180276 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.180515 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.180497 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.187456 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.187437 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdkf8\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-kube-api-access-pdkf8\") pod \"prometheus-k8s-0\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:30.232740 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:30.232711 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:31.126253 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:31.126212 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:31.226378 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:31.226324 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:31.264017 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:31.263712 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-76599cfbd6-w7cnp"] Jul 17 13:20:31.480347 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:31.480314 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20685541_c9ad_4c7c_bc06_21371aff740f.slice/crio-e80bd4ff48da11f8aed6f235f700c8cf6d59fae6d8a28679331c3c6c71c4d365 WatchSource:0}: Error finding container e80bd4ff48da11f8aed6f235f700c8cf6d59fae6d8a28679331c3c6c71c4d365: Status 404 returned error can't find the container with id e80bd4ff48da11f8aed6f235f700c8cf6d59fae6d8a28679331c3c6c71c4d365 Jul 17 13:20:31.678377 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:31.678305 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb"] Jul 17 13:20:31.909930 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:31.908605 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-864cd47f86-fd84k"] Jul 17 13:20:31.918025 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:31.918000 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9dce0e0_9a51_4c2a_8cfc_f91ee6b75866.slice/crio-3c1f965736719915dd33f4650474f0e05721ccd11896c019415638aaadc65bde WatchSource:0}: Error finding container 3c1f965736719915dd33f4650474f0e05721ccd11896c019415638aaadc65bde: Status 404 returned error can't find the container with id 3c1f965736719915dd33f4650474f0e05721ccd11896c019415638aaadc65bde Jul 17 13:20:31.937298 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:31.937265 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-555969c789-bscs4"] Jul 17 13:20:31.939074 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:31.939036 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a664b5a_72aa_4fc3_8738_93d0a51744cc.slice/crio-b7bccadb6402105e6c96a6be9db42e3d9f6da176d38efc65d3b2e83d8399279b WatchSource:0}: Error finding container b7bccadb6402105e6c96a6be9db42e3d9f6da176d38efc65d3b2e83d8399279b: Status 404 returned error can't find the container with id b7bccadb6402105e6c96a6be9db42e3d9f6da176d38efc65d3b2e83d8399279b Jul 17 13:20:31.939165 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:31.939115 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:20:31.940747 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:20:31.940698 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29c4247_b3b1_48e9_b4aa_2eeb44a2b860.slice/crio-e38502a7ad86f93310dcf6e9f9702fa65a9c9b2ed74296dd0c4f9a7245e2c477 WatchSource:0}: Error finding container e38502a7ad86f93310dcf6e9f9702fa65a9c9b2ed74296dd0c4f9a7245e2c477: Status 404 returned error can't find the container with id e38502a7ad86f93310dcf6e9f9702fa65a9c9b2ed74296dd0c4f9a7245e2c477 Jul 17 13:20:32.481869 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.480887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864cd47f86-fd84k" event={"ID":"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866","Type":"ContainerStarted","Data":"586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073"} Jul 17 13:20:32.481869 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.481037 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864cd47f86-fd84k" event={"ID":"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866","Type":"ContainerStarted","Data":"3c1f965736719915dd33f4650474f0e05721ccd11896c019415638aaadc65bde"} Jul 17 13:20:32.487050 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.486100 2560 generic.go:356] "Generic (PLEG): container finished" podID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" exitCode=0 Jul 17 13:20:32.487050 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.486203 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075"} Jul 17 13:20:32.487050 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.486267 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerStarted","Data":"e38502a7ad86f93310dcf6e9f9702fa65a9c9b2ed74296dd0c4f9a7245e2c477"} Jul 17 13:20:32.491705 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.491239 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9b6c9" event={"ID":"b91e071d-432f-4a3d-8f62-f18403981b2d","Type":"ContainerStarted","Data":"ab6d6d8438cbf41ce381bd1ad1bf2cb7b83cd5f88a58690714d4c1510d0e1325"} Jul 17 13:20:32.493627 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.493559 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555969c789-bscs4" event={"ID":"6a664b5a-72aa-4fc3-8738-93d0a51744cc","Type":"ContainerStarted","Data":"b7bccadb6402105e6c96a6be9db42e3d9f6da176d38efc65d3b2e83d8399279b"} Jul 17 13:20:32.497960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.497778 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" event={"ID":"e110e86d-d307-469c-8ce9-1c369156ac8d","Type":"ContainerStarted","Data":"28068eccffb3178d54f91cbc5ed0870b94b94d1de9269e82b6e47649cf1d7c35"} Jul 17 13:20:32.497960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.497842 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" event={"ID":"e110e86d-d307-469c-8ce9-1c369156ac8d","Type":"ContainerStarted","Data":"84f58760c01afdfe5db5860c18dbddce432599f65d03ac3a9b9512da6b3125e0"} Jul 17 13:20:32.497960 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.497882 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" event={"ID":"e110e86d-d307-469c-8ce9-1c369156ac8d","Type":"ContainerStarted","Data":"01948ae8bbd6f281cbe1725ad9378a5d0614a6e9432ff68d986a60e3034b2cac"} Jul 17 13:20:32.499293 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.499249 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-864cd47f86-fd84k" podStartSLOduration=3.499231749 podStartE2EDuration="3.499231749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:32.498361923 +0000 UTC m=+74.576475588" watchObservedRunningTime="2026-07-17 13:20:32.499231749 +0000 UTC m=+74.577345413" Jul 17 13:20:32.501032 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.500988 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" event={"ID":"20685541-c9ad-4c7c-bc06-21371aff740f","Type":"ContainerStarted","Data":"e80bd4ff48da11f8aed6f235f700c8cf6d59fae6d8a28679331c3c6c71c4d365"} Jul 17 13:20:32.504004 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.503980 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8nhwn" event={"ID":"2441c8b2-c301-47f9-96f2-eb0049a7dda3","Type":"ContainerStarted","Data":"cf298a2fca398d3b865dc41b21610a30c053fa4b46f749ed673f2999b04e785e"} Jul 17 13:20:32.504433 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.504413 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:20:32.508173 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.508114 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" event={"ID":"be85d966-ec82-4885-932f-143a63405dbd","Type":"ContainerStarted","Data":"aef766ffe88e30148b28d6a8004148c91d22876702002dd83418303308d2b231"} Jul 17 13:20:32.519195 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.515181 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9b6c9" podStartSLOduration=66.478463776 podStartE2EDuration="1m13.5151649s" totalImagesPullingTime="7.036701124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:32.513535826 +0000 UTC m=+74.591649493" watchObservedRunningTime="2026-07-17 13:20:32.5151649 +0000 UTC m=+74.593278562" Jul 17 13:20:32.519195 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.516206 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerStarted","Data":"d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f"} Jul 17 13:20:32.519195 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.516312 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerStarted","Data":"6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474"} Jul 17 13:20:32.519195 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.516425 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerStarted","Data":"049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0"} Jul 17 13:20:32.519195 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.516474 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerStarted","Data":"642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851"} Jul 17 13:20:32.519195 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.516520 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerStarted","Data":"ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1"} Jul 17 13:20:32.557051 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:32.556536 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8nhwn" podStartSLOduration=66.976346494 podStartE2EDuration="1m13.556516896s" totalImagesPullingTime="6.580170402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:32.554648711 +0000 UTC m=+74.632762373" watchObservedRunningTime="2026-07-17 13:20:32.556516896 +0000 UTC m=+74.634630561" Jul 17 13:20:33.524479 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:33.524438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" event={"ID":"e110e86d-d307-469c-8ce9-1c369156ac8d","Type":"ContainerStarted","Data":"4a225665604bf0b5a59caa6ea410eae3035359216f75c539054d30cbf80ea1cf"} Jul 17 13:20:33.524962 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:33.524531 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" event={"ID":"e110e86d-d307-469c-8ce9-1c369156ac8d","Type":"ContainerStarted","Data":"a176718cc0b2b6b88c9fa44164c440ed619412804645f0bf7f3b6a7f89aea562"} Jul 17 13:20:33.527806 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:33.527779 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerStarted","Data":"0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114"} Jul 17 13:20:33.555075 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:33.555020 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.718884094 podStartE2EDuration="9.55500052s" totalImagesPullingTime="5.803784231s" totalInitContainerRuntime="32.332195ms" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:33.552580029 +0000 UTC m=+75.630693697" watchObservedRunningTime="2026-07-17 13:20:33.55500052 +0000 UTC m=+75.633114185" Jul 17 13:20:34.535434 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.535399 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555969c789-bscs4" event={"ID":"6a664b5a-72aa-4fc3-8738-93d0a51744cc","Type":"ContainerStarted","Data":"37b5cb716e130b99535ea7af393c5cc5e00080845bff60c93170dd53d58ae391"} Jul 17 13:20:34.549132 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.549096 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" event={"ID":"e110e86d-d307-469c-8ce9-1c369156ac8d","Type":"ContainerStarted","Data":"dd0d67fc23f1b4a0a6ac6cb4b6870d9da598599af2c54be3d6fed8b17d2ffc24"} Jul 17 13:20:34.549592 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.549576 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:34.551400 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.551380 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" event={"ID":"20685541-c9ad-4c7c-bc06-21371aff740f","Type":"ContainerStarted","Data":"198f75f42876b604d58d9322dbfe83c77391e8ff9f0953da3e3d90f7c35a83d6"} Jul 17 13:20:34.553829 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.553806 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" event={"ID":"be85d966-ec82-4885-932f-143a63405dbd","Type":"ContainerStarted","Data":"dad331a46918b24efc8a99060490a9c5608d38fe96e7a1cdafb1f9416c22eeb0"} Jul 17 13:20:34.554201 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.554139 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:34.560448 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.560390 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" Jul 17 13:20:34.573121 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.573077 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" podStartSLOduration=3.541241636 podStartE2EDuration="9.573060138s" totalImagesPullingTime="6.031818502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:34.570440101 +0000 UTC m=+76.648553764" watchObservedRunningTime="2026-07-17 13:20:34.573060138 +0000 UTC m=+76.651173806" Jul 17 13:20:34.588345 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.587050 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" podStartSLOduration=3.7106002179999997 podStartE2EDuration="6.587034638s" totalImagesPullingTime="2.87643442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:34.585743622 +0000 UTC m=+76.663857300" watchObservedRunningTime="2026-07-17 13:20:34.587034638 +0000 UTC m=+76.665148304" Jul 17 13:20:34.604011 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:34.603965 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-pr6wb" podStartSLOduration=3.928806742 podStartE2EDuration="6.603946244s" totalImagesPullingTime="2.675139502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:34.601238368 +0000 UTC m=+76.679352036" watchObservedRunningTime="2026-07-17 13:20:34.603946244 +0000 UTC m=+76.682059910" Jul 17 13:20:35.561173 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:35.561136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555969c789-bscs4" event={"ID":"6a664b5a-72aa-4fc3-8738-93d0a51744cc","Type":"ContainerStarted","Data":"03853091543febc667f67d5ae0455616bc07a1776137d2b192ace551e7ebe963"} Jul 17 13:20:35.561643 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:35.561212 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-555969c789-bscs4" event={"ID":"6a664b5a-72aa-4fc3-8738-93d0a51744cc","Type":"ContainerStarted","Data":"40d6e655b5bd3f460a6252bfe911e5bb0bbea6431bb5dfcdee98cf04041c2750"} Jul 17 13:20:35.586370 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:35.584784 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-555969c789-bscs4" podStartSLOduration=5.160583464 podStartE2EDuration="7.584769s" totalImagesPullingTime="2.424185536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:35.582342842 +0000 UTC m=+77.660456506" watchObservedRunningTime="2026-07-17 13:20:35.584769 +0000 UTC m=+77.662882664" Jul 17 13:20:36.390675 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:36.387009 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864cd47f86-fd84k"] Jul 17 13:20:36.446307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:36.446270 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:36.507824 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:36.507796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:36.569134 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:36.569020 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerStarted","Data":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} Jul 17 13:20:36.569134 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:36.569105 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerStarted","Data":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} Jul 17 13:20:37.576549 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:37.576512 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerStarted","Data":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} Jul 17 13:20:37.576882 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:37.576592 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerStarted","Data":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} Jul 17 13:20:37.576882 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:37.576669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerStarted","Data":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} Jul 17 13:20:37.576882 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:37.576710 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerStarted","Data":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} Jul 17 13:20:37.604140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:37.604085 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.664668363 podStartE2EDuration="8.604071032s" totalImagesPullingTime="3.907900283s" totalInitContainerRuntime="31.502386ms" isStatefulPod=false podCreationTimestamp="2026-07-17 13:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:20:37.602344457 +0000 UTC m=+79.680458158" watchObservedRunningTime="2026-07-17 13:20:37.604071032 +0000 UTC m=+79.682184687" Jul 17 13:20:40.084905 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:40.084859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:20:40.233763 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:40.233734 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:20:40.568196 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:40.568097 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5c997d6f94-bf7np" Jul 17 13:20:42.716273 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:42.716229 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:42.768827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:42.768800 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:47.547361 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:47.547320 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f9bbd547d-qrmjz"] Jul 17 13:20:47.647141 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:47.647107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:47.743726 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:47.743694 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:48.348757 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:48.348727 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:48.348936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:48.348813 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:20:50.495557 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:50.495497 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5dfcfd868c-7b46x" podUID="72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" containerName="console" containerID="cri-o://fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8" gracePeriod=15 Jul 17 13:20:51.274334 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.274308 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:51.374964 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.374871 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-oauth-serving-cert\") pod \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " Jul 17 13:20:51.374964 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.374904 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-serving-cert\") pod \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " Jul 17 13:20:51.374964 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.374944 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-config\") pod \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " Jul 17 13:20:51.375170 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.374974 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7wt5\" (UniqueName: \"kubernetes.io/projected/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-kube-api-access-d7wt5\") pod \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " Jul 17 13:20:51.375170 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.375024 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-oauth-config\") pod \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " Jul 17 13:20:51.375170 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.375063 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-service-ca\") pod \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\" (UID: \"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82\") " Jul 17 13:20:51.375344 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.375318 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" (UID: "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:20:51.375413 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.375326 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-config" (OuterVolumeSpecName: "console-config") pod "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" (UID: "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:20:51.375629 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.375606 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-service-ca" (OuterVolumeSpecName: "service-ca") pod "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" (UID: "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:20:51.377350 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.377324 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" (UID: "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:20:51.377467 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.377384 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-kube-api-access-d7wt5" (OuterVolumeSpecName: "kube-api-access-d7wt5") pod "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" (UID: "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82"). InnerVolumeSpecName "kube-api-access-d7wt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:20:51.377467 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.377421 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" (UID: "72f9fc6d-a9ad-4731-8f01-f2dda6a62a82"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:20:51.476371 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.476346 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7wt5\" (UniqueName: \"kubernetes.io/projected/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-kube-api-access-d7wt5\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:20:51.476371 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.476368 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-oauth-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:20:51.476503 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.476377 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-service-ca\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:20:51.476503 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.476386 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-oauth-serving-cert\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:20:51.476503 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.476394 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-serving-cert\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:20:51.476503 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.476403 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82-console-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:20:51.637242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.637171 2560 generic.go:356] "Generic (PLEG): container finished" podID="72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" containerID="fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8" exitCode=0 Jul 17 13:20:51.637242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.637222 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dfcfd868c-7b46x" Jul 17 13:20:51.637242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.637231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfcfd868c-7b46x" event={"ID":"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82","Type":"ContainerDied","Data":"fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8"} Jul 17 13:20:51.637693 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.637297 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dfcfd868c-7b46x" event={"ID":"72f9fc6d-a9ad-4731-8f01-f2dda6a62a82","Type":"ContainerDied","Data":"91ff7bb13e31c476306881b161a7ec6a22444c8af57929d781795f8a55eae9e8"} Jul 17 13:20:51.637693 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.637324 2560 scope.go:117] "RemoveContainer" containerID="fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8" Jul 17 13:20:51.645976 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.645956 2560 scope.go:117] "RemoveContainer" containerID="fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8" Jul 17 13:20:51.646234 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:20:51.646216 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8\": container with ID starting with fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8 not found: ID does not exist" containerID="fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8" Jul 17 13:20:51.646285 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.646241 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8"} err="failed to get container status \"fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8\": rpc error: code = NotFound desc = could not find container \"fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8\": container with ID starting with fc40aae16b08bd39ca390609b2a67761d17a91d26c88e58dce6c6468025c32e8 not found: ID does not exist" Jul 17 13:20:51.657706 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.657686 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5dfcfd868c-7b46x"] Jul 17 13:20:51.664353 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:51.664333 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5dfcfd868c-7b46x"] Jul 17 13:20:53.101242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:53.101213 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" path="/var/lib/kubelet/pods/72f9fc6d-a9ad-4731-8f01-f2dda6a62a82/volumes" Jul 17 13:20:54.117754 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:54.117707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:54.171117 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:54.171082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:57.711603 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:57.711570 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:20:57.764725 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:20:57.764693 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:01.412716 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.412657 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-864cd47f86-fd84k" podUID="a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" containerName="console" containerID="cri-o://586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073" gracePeriod=15 Jul 17 13:21:01.654303 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.654280 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:21:01.679936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.678682 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" containerID="586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073" exitCode=0 Jul 17 13:21:01.679936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.678776 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864cd47f86-fd84k" event={"ID":"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866","Type":"ContainerDied","Data":"586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073"} Jul 17 13:21:01.679936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.678870 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-864cd47f86-fd84k" event={"ID":"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866","Type":"ContainerDied","Data":"3c1f965736719915dd33f4650474f0e05721ccd11896c019415638aaadc65bde"} Jul 17 13:21:01.679936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.678905 2560 scope.go:117] "RemoveContainer" containerID="586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073" Jul 17 13:21:01.679936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.678961 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-864cd47f86-fd84k" Jul 17 13:21:01.691827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.691801 2560 scope.go:117] "RemoveContainer" containerID="586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073" Jul 17 13:21:01.692123 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:01.692102 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073\": container with ID starting with 586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073 not found: ID does not exist" containerID="586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073" Jul 17 13:21:01.692194 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.692131 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073"} err="failed to get container status \"586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073\": rpc error: code = NotFound desc = could not find container \"586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073\": container with ID starting with 586b4a4c80d2b353a2982438ff2c3c2c2b914ade3ccb9ba84ef4fc479c1aa073 not found: ID does not exist" Jul 17 13:21:01.762152 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762126 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-service-ca\") pod \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " Jul 17 13:21:01.762269 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762156 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-oauth-serving-cert\") pod \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " Jul 17 13:21:01.762269 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762180 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d6vj2\" (UniqueName: \"kubernetes.io/projected/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-kube-api-access-d6vj2\") pod \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " Jul 17 13:21:01.762269 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762199 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-serving-cert\") pod \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " Jul 17 13:21:01.762426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762272 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-trusted-ca-bundle\") pod \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " Jul 17 13:21:01.762426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762328 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-config\") pod \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " Jul 17 13:21:01.762426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762360 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-oauth-config\") pod \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\" (UID: \"a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866\") " Jul 17 13:21:01.762579 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762523 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-service-ca" (OuterVolumeSpecName: "service-ca") pod "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" (UID: "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:01.762678 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762657 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" (UID: "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:01.762737 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762665 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-service-ca\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:01.762737 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762712 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-config" (OuterVolumeSpecName: "console-config") pod "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" (UID: "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:01.762905 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.762885 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" (UID: "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:01.764391 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.764362 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" (UID: "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:01.764476 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.764412 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-kube-api-access-d6vj2" (OuterVolumeSpecName: "kube-api-access-d6vj2") pod "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" (UID: "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866"). InnerVolumeSpecName "kube-api-access-d6vj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:21:01.764476 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.764418 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" (UID: "a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:01.863929 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.863884 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:01.863929 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.863927 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-oauth-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:01.864090 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.863937 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-oauth-serving-cert\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:01.864090 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.863946 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6vj2\" (UniqueName: \"kubernetes.io/projected/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-kube-api-access-d6vj2\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:01.864090 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.863955 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-console-serving-cert\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:01.864090 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.863963 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866-trusted-ca-bundle\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:01.999639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:01.999607 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-864cd47f86-fd84k"] Jul 17 13:21:02.003295 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:02.003273 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-864cd47f86-fd84k"] Jul 17 13:21:03.100297 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:03.100264 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" path="/var/lib/kubelet/pods/a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866/volumes" Jul 17 13:21:04.557269 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:04.557240 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8nhwn" Jul 17 13:21:08.354162 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:08.354134 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:21:08.357879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:08.357855 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-76599cfbd6-w7cnp" Jul 17 13:21:12.569129 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.569084 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f9bbd547d-qrmjz" podUID="c7386392-9f3b-4ea9-a1d4-d273e7319bee" containerName="console" containerID="cri-o://b22b4ae8f6a867bd0f17e8a68cb2d12bd1057ecd93758379594821fd21ac7953" gracePeriod=15 Jul 17 13:21:12.729970 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.729890 2560 generic.go:356] "Generic (PLEG): container finished" podID="c7386392-9f3b-4ea9-a1d4-d273e7319bee" containerID="b22b4ae8f6a867bd0f17e8a68cb2d12bd1057ecd93758379594821fd21ac7953" exitCode=0 Jul 17 13:21:12.729970 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.729961 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9bbd547d-qrmjz" event={"ID":"c7386392-9f3b-4ea9-a1d4-d273e7319bee","Type":"ContainerDied","Data":"b22b4ae8f6a867bd0f17e8a68cb2d12bd1057ecd93758379594821fd21ac7953"} Jul 17 13:21:12.806542 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.806521 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:21:12.957858 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.957832 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-trusted-ca-bundle\") pod \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " Jul 17 13:21:12.958023 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.957867 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-serving-cert\") pod \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " Jul 17 13:21:12.958023 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.957883 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-oauth-serving-cert\") pod \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " Jul 17 13:21:12.958023 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.957936 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-config\") pod \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " Jul 17 13:21:12.958129 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958105 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-service-ca\") pod \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " Jul 17 13:21:12.958179 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958168 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-oauth-config\") pod \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " Jul 17 13:21:12.958250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958224 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47rtj\" (UniqueName: \"kubernetes.io/projected/c7386392-9f3b-4ea9-a1d4-d273e7319bee-kube-api-access-47rtj\") pod \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\" (UID: \"c7386392-9f3b-4ea9-a1d4-d273e7319bee\") " Jul 17 13:21:12.958394 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958294 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c7386392-9f3b-4ea9-a1d4-d273e7319bee" (UID: "c7386392-9f3b-4ea9-a1d4-d273e7319bee"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:12.958457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958326 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-config" (OuterVolumeSpecName: "console-config") pod "c7386392-9f3b-4ea9-a1d4-d273e7319bee" (UID: "c7386392-9f3b-4ea9-a1d4-d273e7319bee"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:12.958457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958358 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c7386392-9f3b-4ea9-a1d4-d273e7319bee" (UID: "c7386392-9f3b-4ea9-a1d4-d273e7319bee"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:12.958560 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958483 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-service-ca" (OuterVolumeSpecName: "service-ca") pod "c7386392-9f3b-4ea9-a1d4-d273e7319bee" (UID: "c7386392-9f3b-4ea9-a1d4-d273e7319bee"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:12.958619 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958566 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-service-ca\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:12.958619 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958580 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-trusted-ca-bundle\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:12.958619 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958590 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-oauth-serving-cert\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:12.958619 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.958599 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:12.960423 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.960399 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7386392-9f3b-4ea9-a1d4-d273e7319bee-kube-api-access-47rtj" (OuterVolumeSpecName: "kube-api-access-47rtj") pod "c7386392-9f3b-4ea9-a1d4-d273e7319bee" (UID: "c7386392-9f3b-4ea9-a1d4-d273e7319bee"). InnerVolumeSpecName "kube-api-access-47rtj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:21:12.960674 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.960647 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c7386392-9f3b-4ea9-a1d4-d273e7319bee" (UID: "c7386392-9f3b-4ea9-a1d4-d273e7319bee"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:12.960765 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:12.960709 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c7386392-9f3b-4ea9-a1d4-d273e7319bee" (UID: "c7386392-9f3b-4ea9-a1d4-d273e7319bee"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:13.059683 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.059644 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-oauth-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:13.059683 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.059679 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47rtj\" (UniqueName: \"kubernetes.io/projected/c7386392-9f3b-4ea9-a1d4-d273e7319bee-kube-api-access-47rtj\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:13.059683 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.059689 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c7386392-9f3b-4ea9-a1d4-d273e7319bee-console-serving-cert\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:13.736023 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.735990 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f9bbd547d-qrmjz" event={"ID":"c7386392-9f3b-4ea9-a1d4-d273e7319bee","Type":"ContainerDied","Data":"b2c4f6b60b7466e5141183bef5836716a248dd98ee78b19c30719586207f1c1c"} Jul 17 13:21:13.736023 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.736018 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f9bbd547d-qrmjz" Jul 17 13:21:13.736550 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.736050 2560 scope.go:117] "RemoveContainer" containerID="b22b4ae8f6a867bd0f17e8a68cb2d12bd1057ecd93758379594821fd21ac7953" Jul 17 13:21:13.754015 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.753993 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f9bbd547d-qrmjz"] Jul 17 13:21:13.757103 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:13.757085 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f9bbd547d-qrmjz"] Jul 17 13:21:15.100457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:15.100426 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7386392-9f3b-4ea9-a1d4-d273e7319bee" path="/var/lib/kubelet/pods/c7386392-9f3b-4ea9-a1d4-d273e7319bee/volumes" Jul 17 13:21:30.233464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:30.233428 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:30.252472 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:30.252451 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:30.821170 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:30.821146 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:31.875768 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:31.875732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:39.754747 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:39.754706 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:39.807357 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:39.807323 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:44.011847 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.011802 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:21:44.012377 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.012185 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="alertmanager" containerID="cri-o://ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1" gracePeriod=120 Jul 17 13:21:44.012377 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.012250 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-metric" containerID="cri-o://d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f" gracePeriod=120 Jul 17 13:21:44.012377 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.012283 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-web" containerID="cri-o://049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0" gracePeriod=120 Jul 17 13:21:44.012377 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.012289 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy" containerID="cri-o://6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474" gracePeriod=120 Jul 17 13:21:44.012563 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.012379 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="config-reloader" containerID="cri-o://642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851" gracePeriod=120 Jul 17 13:21:44.012563 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.012392 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="prom-label-proxy" containerID="cri-o://0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114" gracePeriod=120 Jul 17 13:21:44.861759 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861731 2560 generic.go:356] "Generic (PLEG): container finished" podID="a986e594-7bc6-4a3c-871b-c821e4206544" containerID="0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114" exitCode=0 Jul 17 13:21:44.861759 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861755 2560 generic.go:356] "Generic (PLEG): container finished" podID="a986e594-7bc6-4a3c-871b-c821e4206544" containerID="d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f" exitCode=0 Jul 17 13:21:44.861759 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861761 2560 generic.go:356] "Generic (PLEG): container finished" podID="a986e594-7bc6-4a3c-871b-c821e4206544" containerID="6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474" exitCode=0 Jul 17 13:21:44.861759 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861766 2560 generic.go:356] "Generic (PLEG): container finished" podID="a986e594-7bc6-4a3c-871b-c821e4206544" containerID="642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851" exitCode=0 Jul 17 13:21:44.862029 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861772 2560 generic.go:356] "Generic (PLEG): container finished" podID="a986e594-7bc6-4a3c-871b-c821e4206544" containerID="ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1" exitCode=0 Jul 17 13:21:44.862029 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114"} Jul 17 13:21:44.862029 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861865 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f"} Jul 17 13:21:44.862029 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861891 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474"} Jul 17 13:21:44.862029 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861931 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851"} Jul 17 13:21:44.862029 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:44.861956 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1"} Jul 17 13:21:45.262868 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.262844 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:45.338280 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338247 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-config-volume\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338295 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-main-db\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338311 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-config-out\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338327 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-tls-assets\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338358 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-main-tls\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338389 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-web-config\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338544 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-cluster-tls-config\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338617 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-web\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338657 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-trusted-ca-bundle\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338844 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338695 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338844 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338721 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpvxj\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-kube-api-access-vpvxj\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338844 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338745 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-metrics-client-ca\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.338844 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.338774 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-metric\") pod \"a986e594-7bc6-4a3c-871b-c821e4206544\" (UID: \"a986e594-7bc6-4a3c-871b-c821e4206544\") " Jul 17 13:21:45.339648 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.339617 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:21:45.340057 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.340028 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:45.340140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.340071 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:45.341798 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.341769 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-config-volume" (OuterVolumeSpecName: "config-volume") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:45.342128 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.342091 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-config-out" (OuterVolumeSpecName: "config-out") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:21:45.342426 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.342303 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:45.342557 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.342516 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:45.342702 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.342678 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:45.343804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.343771 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:45.343903 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.343847 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-kube-api-access-vpvxj" (OuterVolumeSpecName: "kube-api-access-vpvxj") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "kube-api-access-vpvxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:21:45.344029 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.344000 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:21:45.346336 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.346314 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:45.352870 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.352840 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-web-config" (OuterVolumeSpecName: "web-config") pod "a986e594-7bc6-4a3c-871b-c821e4206544" (UID: "a986e594-7bc6-4a3c-871b-c821e4206544"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:45.440055 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440029 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-config-volume\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440055 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440052 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-main-db\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440062 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a986e594-7bc6-4a3c-871b-c821e4206544-config-out\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440072 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-tls-assets\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440080 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-main-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440089 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-web-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440098 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-cluster-tls-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440107 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440117 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440126 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440134 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpvxj\" (UniqueName: \"kubernetes.io/projected/a986e594-7bc6-4a3c-871b-c821e4206544-kube-api-access-vpvxj\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440142 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a986e594-7bc6-4a3c-871b-c821e4206544-metrics-client-ca\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.440187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.440150 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/a986e594-7bc6-4a3c-871b-c821e4206544-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:45.868894 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.868821 2560 generic.go:356] "Generic (PLEG): container finished" podID="a986e594-7bc6-4a3c-871b-c821e4206544" containerID="049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0" exitCode=0 Jul 17 13:21:45.868894 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.868860 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0"} Jul 17 13:21:45.869079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.868927 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"a986e594-7bc6-4a3c-871b-c821e4206544","Type":"ContainerDied","Data":"b3e8248f917647120ef06fbf46b8b059d695774c9dc6004128d8c66cb6239987"} Jul 17 13:21:45.869079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.868939 2560 scope.go:117] "RemoveContainer" containerID="0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114" Jul 17 13:21:45.869079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.868926 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:45.877880 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.877500 2560 scope.go:117] "RemoveContainer" containerID="d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f" Jul 17 13:21:45.885003 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.884982 2560 scope.go:117] "RemoveContainer" containerID="6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474" Jul 17 13:21:45.891394 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.891376 2560 scope.go:117] "RemoveContainer" containerID="049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0" Jul 17 13:21:45.894328 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.894309 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:21:45.898587 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.898565 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:21:45.898998 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.898981 2560 scope.go:117] "RemoveContainer" containerID="642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851" Jul 17 13:21:45.905462 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.905440 2560 scope.go:117] "RemoveContainer" containerID="ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1" Jul 17 13:21:45.912209 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.912192 2560 scope.go:117] "RemoveContainer" containerID="e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed" Jul 17 13:21:45.918808 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.918795 2560 scope.go:117] "RemoveContainer" containerID="0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114" Jul 17 13:21:45.919074 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:45.919057 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114\": container with ID starting with 0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114 not found: ID does not exist" containerID="0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114" Jul 17 13:21:45.919131 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919085 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114"} err="failed to get container status \"0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114\": rpc error: code = NotFound desc = could not find container \"0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114\": container with ID starting with 0aca964dc46b97fe8b0a5fdf884a9b35795dd267eed8eb3ea9b15d2c7ae2f114 not found: ID does not exist" Jul 17 13:21:45.919131 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919103 2560 scope.go:117] "RemoveContainer" containerID="d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f" Jul 17 13:21:45.919349 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:45.919332 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f\": container with ID starting with d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f not found: ID does not exist" containerID="d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f" Jul 17 13:21:45.919417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919358 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f"} err="failed to get container status \"d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f\": rpc error: code = NotFound desc = could not find container \"d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f\": container with ID starting with d522f73857bfc95ec2ac0b5781152c902a322a5b57a0b37d77b28cda88891a5f not found: ID does not exist" Jul 17 13:21:45.919417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919381 2560 scope.go:117] "RemoveContainer" containerID="6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474" Jul 17 13:21:45.919607 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:45.919590 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474\": container with ID starting with 6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474 not found: ID does not exist" containerID="6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474" Jul 17 13:21:45.919646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919613 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474"} err="failed to get container status \"6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474\": rpc error: code = NotFound desc = could not find container \"6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474\": container with ID starting with 6a3ad1347562bd811d3598b134a2da43ebd38b07605ed19dce70f90af1bbf474 not found: ID does not exist" Jul 17 13:21:45.919646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919627 2560 scope.go:117] "RemoveContainer" containerID="049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0" Jul 17 13:21:45.919843 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:45.919827 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0\": container with ID starting with 049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0 not found: ID does not exist" containerID="049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0" Jul 17 13:21:45.919883 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919847 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0"} err="failed to get container status \"049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0\": rpc error: code = NotFound desc = could not find container \"049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0\": container with ID starting with 049451d6b8528d457d23db1972f543e175a9a2e705a6cea4f46b53dcd60565b0 not found: ID does not exist" Jul 17 13:21:45.919883 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.919860 2560 scope.go:117] "RemoveContainer" containerID="642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851" Jul 17 13:21:45.920106 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:45.920090 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851\": container with ID starting with 642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851 not found: ID does not exist" containerID="642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851" Jul 17 13:21:45.920149 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.920109 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851"} err="failed to get container status \"642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851\": rpc error: code = NotFound desc = could not find container \"642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851\": container with ID starting with 642b2d4e8ca2ea6842309f16b05359b9dac9051a1ea9b653a2ec91475f35f851 not found: ID does not exist" Jul 17 13:21:45.920149 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.920123 2560 scope.go:117] "RemoveContainer" containerID="ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1" Jul 17 13:21:45.920312 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:45.920297 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1\": container with ID starting with ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1 not found: ID does not exist" containerID="ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1" Jul 17 13:21:45.920351 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.920315 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1"} err="failed to get container status \"ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1\": rpc error: code = NotFound desc = could not find container \"ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1\": container with ID starting with ef9ccc10bb0dd009fa68f79f804b8e2894a1135c969979725faf4cab516aeee1 not found: ID does not exist" Jul 17 13:21:45.920351 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.920327 2560 scope.go:117] "RemoveContainer" containerID="e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed" Jul 17 13:21:45.920559 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:45.920544 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed\": container with ID starting with e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed not found: ID does not exist" containerID="e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed" Jul 17 13:21:45.920606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.920562 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed"} err="failed to get container status \"e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed\": rpc error: code = NotFound desc = could not find container \"e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed\": container with ID starting with e0967cebff0382df97dc9e405bc8d8bcb8e69a4b6a5ab2fd513e908feb67b0ed not found: ID does not exist" Jul 17 13:21:45.923400 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923380 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:21:45.923689 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923676 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy" Jul 17 13:21:45.923734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923692 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy" Jul 17 13:21:45.923734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923706 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="config-reloader" Jul 17 13:21:45.923734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923712 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="config-reloader" Jul 17 13:21:45.923734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923719 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" containerName="console" Jul 17 13:21:45.923734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923724 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" containerName="console" Jul 17 13:21:45.923734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923729 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7386392-9f3b-4ea9-a1d4-d273e7319bee" containerName="console" Jul 17 13:21:45.923734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923734 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7386392-9f3b-4ea9-a1d4-d273e7319bee" containerName="console" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923744 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" containerName="console" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923749 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" containerName="console" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923758 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-metric" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923762 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-metric" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923774 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="init-config-reloader" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923782 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="init-config-reloader" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923795 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="alertmanager" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923800 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="alertmanager" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923806 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="prom-label-proxy" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923810 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="prom-label-proxy" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923819 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-web" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923824 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-web" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923873 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923883 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-metric" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923888 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f9fc6d-a9ad-4731-8f01-f2dda6a62a82" containerName="console" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923895 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="alertmanager" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923900 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9dce0e0-9a51-4c2a-8cfc-f91ee6b75866" containerName="console" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923917 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="config-reloader" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923923 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7386392-9f3b-4ea9-a1d4-d273e7319bee" containerName="console" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923930 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="kube-rbac-proxy-web" Jul 17 13:21:45.923951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.923937 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" containerName="prom-label-proxy" Jul 17 13:21:45.927812 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.927795 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:45.930493 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930473 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 17 13:21:45.930493 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930483 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 17 13:21:45.930644 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930492 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 17 13:21:45.930644 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930626 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 17 13:21:45.930644 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930637 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 17 13:21:45.930821 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930693 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 17 13:21:45.930821 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930711 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 17 13:21:45.930821 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.930720 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 17 13:21:45.931137 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.931124 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hg5f6\"" Jul 17 13:21:45.941951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.940077 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 17 13:21:45.943968 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:45.943948 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:21:46.044666 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044635 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4xff\" (UniqueName: \"kubernetes.io/projected/103c6d6f-a6a4-45f1-a96f-ac72496052cc-kube-api-access-h4xff\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.044820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044670 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/103c6d6f-a6a4-45f1-a96f-ac72496052cc-config-out\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.044820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.044820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044712 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.044820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044756 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/103c6d6f-a6a4-45f1-a96f-ac72496052cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.044820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044775 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/103c6d6f-a6a4-45f1-a96f-ac72496052cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.044820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044819 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.045038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044835 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/103c6d6f-a6a4-45f1-a96f-ac72496052cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.045038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044852 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-web-config\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.045038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044880 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.045038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044894 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.045038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.045038 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.044957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/103c6d6f-a6a4-45f1-a96f-ac72496052cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146254 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146272 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/103c6d6f-a6a4-45f1-a96f-ac72496052cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146296 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4xff\" (UniqueName: \"kubernetes.io/projected/103c6d6f-a6a4-45f1-a96f-ac72496052cc-kube-api-access-h4xff\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/103c6d6f-a6a4-45f1-a96f-ac72496052cc-config-out\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146388 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/103c6d6f-a6a4-45f1-a96f-ac72496052cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146755 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/103c6d6f-a6a4-45f1-a96f-ac72496052cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146755 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146578 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146755 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146607 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/103c6d6f-a6a4-45f1-a96f-ac72496052cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.146755 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.146639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-web-config\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.147172 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.147146 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/103c6d6f-a6a4-45f1-a96f-ac72496052cc-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149140 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/103c6d6f-a6a4-45f1-a96f-ac72496052cc-config-out\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149338 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149348 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149472 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-config-volume\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149603 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/103c6d6f-a6a4-45f1-a96f-ac72496052cc-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149630 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-web-config\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.149996 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.149953 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/103c6d6f-a6a4-45f1-a96f-ac72496052cc-tls-assets\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.150261 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.150240 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.150309 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.150291 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/103c6d6f-a6a4-45f1-a96f-ac72496052cc-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.151256 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.151239 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/103c6d6f-a6a4-45f1-a96f-ac72496052cc-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.155240 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.155221 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4xff\" (UniqueName: \"kubernetes.io/projected/103c6d6f-a6a4-45f1-a96f-ac72496052cc-kube-api-access-h4xff\") pod \"alertmanager-main-0\" (UID: \"103c6d6f-a6a4-45f1-a96f-ac72496052cc\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.239946 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.239924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 13:21:46.370608 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.370586 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 13:21:46.371687 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:21:46.371665 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod103c6d6f_a6a4_45f1_a96f_ac72496052cc.slice/crio-1a64ed4446a57346beb7f46b373ebe0fc6693db8ed997990d30f8c6fe93319fa WatchSource:0}: Error finding container 1a64ed4446a57346beb7f46b373ebe0fc6693db8ed997990d30f8c6fe93319fa: Status 404 returned error can't find the container with id 1a64ed4446a57346beb7f46b373ebe0fc6693db8ed997990d30f8c6fe93319fa Jul 17 13:21:46.874607 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.874572 2560 generic.go:356] "Generic (PLEG): container finished" podID="103c6d6f-a6a4-45f1-a96f-ac72496052cc" containerID="58a68b1a21c3cf22caa5ee0b33b38356078750b93b8818d13c45a201e03bb8f0" exitCode=0 Jul 17 13:21:46.874768 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.874658 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerDied","Data":"58a68b1a21c3cf22caa5ee0b33b38356078750b93b8818d13c45a201e03bb8f0"} Jul 17 13:21:46.874768 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:46.874727 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerStarted","Data":"1a64ed4446a57346beb7f46b373ebe0fc6693db8ed997990d30f8c6fe93319fa"} Jul 17 13:21:47.102562 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.102533 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a986e594-7bc6-4a3c-871b-c821e4206544" path="/var/lib/kubelet/pods/a986e594-7bc6-4a3c-871b-c821e4206544/volumes" Jul 17 13:21:47.882058 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.882021 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerStarted","Data":"1f274d1e3f36ed8f202db3a1b0ae779193e9f221aa37afa84d86b9049e3e13f2"} Jul 17 13:21:47.882417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.882105 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerStarted","Data":"a46da2c8ce997bf2d27a339b907c02fd3e8b7e17a83f5997755cded429751299"} Jul 17 13:21:47.882417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.882146 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerStarted","Data":"6cc42e7e62bb0aa968175ffd0bbb020506468dde8240f5a2713bfb9998a27c8c"} Jul 17 13:21:47.882417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.882185 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerStarted","Data":"caa75ae7d51298f183b82b8747c5a5c8eca226a1a9dbbdfbf3a5d718a5371837"} Jul 17 13:21:47.882417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.882222 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerStarted","Data":"199dcf01c494f849ace3c11fcd36a29a2d89bc64007ca3d0ddef02b09b037c66"} Jul 17 13:21:47.882417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.882255 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"103c6d6f-a6a4-45f1-a96f-ac72496052cc","Type":"ContainerStarted","Data":"9d550cf85ba632361bd7c3513eb75a6c1ff752220411ba67001c3a8606275615"} Jul 17 13:21:47.908452 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:47.908370 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.89297622 podStartE2EDuration="2.908348466s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.372246ms" isStatefulPod=false podCreationTimestamp="2026-07-17 13:21:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:21:47.90639843 +0000 UTC m=+149.984512129" watchObservedRunningTime="2026-07-17 13:21:47.908348466 +0000 UTC m=+149.986462131" Jul 17 13:21:48.253205 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.253176 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:21:48.253686 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.253495 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="prometheus" containerID="cri-o://c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" gracePeriod=600 Jul 17 13:21:48.253686 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.253520 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy" containerID="cri-o://f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" gracePeriod=600 Jul 17 13:21:48.253686 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.253537 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-thanos" containerID="cri-o://b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" gracePeriod=600 Jul 17 13:21:48.253686 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.253582 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-web" containerID="cri-o://1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" gracePeriod=600 Jul 17 13:21:48.253686 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.253541 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="thanos-sidecar" containerID="cri-o://053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" gracePeriod=600 Jul 17 13:21:48.254055 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.253674 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="config-reloader" containerID="cri-o://936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" gracePeriod=600 Jul 17 13:21:48.501755 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.501723 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:48.568897 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.568817 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-web-config\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.568897 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.568867 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569168 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.568902 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-serving-certs-ca-bundle\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569168 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.568943 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-1\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569168 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.568979 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-thanos-prometheus-http-client-file\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569168 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569092 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-tls\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569168 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569139 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-tls-assets\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569198 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdkf8\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-kube-api-access-pdkf8\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569229 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-db\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569264 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-0\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569272 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569301 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-kube-rbac-proxy\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569332 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-metrics-client-ca\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569364 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-trusted-ca-bundle\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569432 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569394 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-metrics-client-certs\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569450 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-2\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569484 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-kubelet-serving-ca-bundle\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569509 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-grpc-tls\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569540 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569584 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config-out\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.569879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569626 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config\") pod \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\" (UID: \"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860\") " Jul 17 13:21:48.570214 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569953 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.571550 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.569327 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:48.571550 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.570344 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:48.571550 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.570558 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:48.571806 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.571660 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:21:48.572076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.572049 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:48.572202 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.572087 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.572271 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.572247 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.572323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.572279 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:48.572389 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.572370 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:21:48.572824 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.572788 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-kube-api-access-pdkf8" (OuterVolumeSpecName: "kube-api-access-pdkf8") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "kube-api-access-pdkf8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:21:48.573501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.573467 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.574025 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.573974 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config" (OuterVolumeSpecName: "config") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.574429 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.574404 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:21:48.574505 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.574410 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.574665 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.574647 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.574737 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.574719 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.575745 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.575721 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.575834 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.575818 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config-out" (OuterVolumeSpecName: "config-out") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:21:48.583772 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.583753 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-web-config" (OuterVolumeSpecName: "web-config") pod "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" (UID: "c29c4247-b3b1-48e9-b4aa-2eeb44a2b860"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:21:48.670625 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670585 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670637 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670662 2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-grpc-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670687 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670711 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config-out\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670727 2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670744 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-web-config\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670760 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670775 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670790 2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-thanos-prometheus-http-client-file\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670808 2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-prometheus-k8s-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670821 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-tls-assets\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670837 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdkf8\" (UniqueName: \"kubernetes.io/projected/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-kube-api-access-pdkf8\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670850 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-db\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670869 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.670891 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670883 2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-kube-rbac-proxy\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.671206 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670901 2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-configmap-metrics-client-ca\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.671206 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670932 2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-prometheus-trusted-ca-bundle\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.671206 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.670947 2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860-secret-metrics-client-certs\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:21:48.889820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889740 2560 generic.go:356] "Generic (PLEG): container finished" podID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" exitCode=0 Jul 17 13:21:48.889820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889763 2560 generic.go:356] "Generic (PLEG): container finished" podID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" exitCode=0 Jul 17 13:21:48.889820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889770 2560 generic.go:356] "Generic (PLEG): container finished" podID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" exitCode=0 Jul 17 13:21:48.889820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889776 2560 generic.go:356] "Generic (PLEG): container finished" podID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" exitCode=0 Jul 17 13:21:48.889820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889782 2560 generic.go:356] "Generic (PLEG): container finished" podID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" exitCode=0 Jul 17 13:21:48.889820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889787 2560 generic.go:356] "Generic (PLEG): container finished" podID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" exitCode=0 Jul 17 13:21:48.889820 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889820 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.889969 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.890018 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.890059 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.890100 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.890036 2560 scope.go:117] "RemoveContainer" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.890140 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} Jul 17 13:21:48.890553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.890181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c29c4247-b3b1-48e9-b4aa-2eeb44a2b860","Type":"ContainerDied","Data":"e38502a7ad86f93310dcf6e9f9702fa65a9c9b2ed74296dd0c4f9a7245e2c477"} Jul 17 13:21:48.898685 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.898664 2560 scope.go:117] "RemoveContainer" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.905560 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.905546 2560 scope.go:117] "RemoveContainer" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.912579 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.912479 2560 scope.go:117] "RemoveContainer" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.914374 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.914353 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:21:48.919568 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.919548 2560 scope.go:117] "RemoveContainer" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.921107 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.921086 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:21:48.926073 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.926056 2560 scope.go:117] "RemoveContainer" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.932993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.932975 2560 scope.go:117] "RemoveContainer" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.939442 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.939425 2560 scope.go:117] "RemoveContainer" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.939676 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:48.939658 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": container with ID starting with b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb not found: ID does not exist" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.939718 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.939684 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} err="failed to get container status \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": rpc error: code = NotFound desc = could not find container \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": container with ID starting with b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb not found: ID does not exist" Jul 17 13:21:48.939718 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.939700 2560 scope.go:117] "RemoveContainer" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.939941 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:48.939904 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": container with ID starting with f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3 not found: ID does not exist" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.940041 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.939941 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} err="failed to get container status \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": rpc error: code = NotFound desc = could not find container \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": container with ID starting with f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3 not found: ID does not exist" Jul 17 13:21:48.940041 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.939954 2560 scope.go:117] "RemoveContainer" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.940168 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:48.940150 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": container with ID starting with 1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2 not found: ID does not exist" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.940205 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940172 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} err="failed to get container status \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": rpc error: code = NotFound desc = could not find container \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": container with ID starting with 1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2 not found: ID does not exist" Jul 17 13:21:48.940205 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940186 2560 scope.go:117] "RemoveContainer" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.940416 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:48.940398 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": container with ID starting with 053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939 not found: ID does not exist" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.940453 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940423 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} err="failed to get container status \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": rpc error: code = NotFound desc = could not find container \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": container with ID starting with 053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939 not found: ID does not exist" Jul 17 13:21:48.940453 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940439 2560 scope.go:117] "RemoveContainer" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.940630 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:48.940615 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": container with ID starting with 936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd not found: ID does not exist" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.940666 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940634 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} err="failed to get container status \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": rpc error: code = NotFound desc = could not find container \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": container with ID starting with 936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd not found: ID does not exist" Jul 17 13:21:48.940666 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940645 2560 scope.go:117] "RemoveContainer" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.940821 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:48.940806 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": container with ID starting with c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f not found: ID does not exist" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.940862 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940824 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} err="failed to get container status \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": rpc error: code = NotFound desc = could not find container \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": container with ID starting with c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f not found: ID does not exist" Jul 17 13:21:48.940862 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.940835 2560 scope.go:117] "RemoveContainer" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.941099 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:21:48.941083 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": container with ID starting with b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075 not found: ID does not exist" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.941149 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941101 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075"} err="failed to get container status \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": rpc error: code = NotFound desc = could not find container \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": container with ID starting with b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075 not found: ID does not exist" Jul 17 13:21:48.941149 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941112 2560 scope.go:117] "RemoveContainer" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.941320 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941305 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} err="failed to get container status \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": rpc error: code = NotFound desc = could not find container \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": container with ID starting with b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb not found: ID does not exist" Jul 17 13:21:48.941362 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941320 2560 scope.go:117] "RemoveContainer" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.941500 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941485 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} err="failed to get container status \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": rpc error: code = NotFound desc = could not find container \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": container with ID starting with f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3 not found: ID does not exist" Jul 17 13:21:48.941542 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941499 2560 scope.go:117] "RemoveContainer" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.941709 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941692 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} err="failed to get container status \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": rpc error: code = NotFound desc = could not find container \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": container with ID starting with 1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2 not found: ID does not exist" Jul 17 13:21:48.941709 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941708 2560 scope.go:117] "RemoveContainer" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.941900 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941883 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} err="failed to get container status \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": rpc error: code = NotFound desc = could not find container \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": container with ID starting with 053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939 not found: ID does not exist" Jul 17 13:21:48.941977 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.941900 2560 scope.go:117] "RemoveContainer" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.942124 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942105 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} err="failed to get container status \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": rpc error: code = NotFound desc = could not find container \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": container with ID starting with 936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd not found: ID does not exist" Jul 17 13:21:48.942178 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942125 2560 scope.go:117] "RemoveContainer" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.942353 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942338 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} err="failed to get container status \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": rpc error: code = NotFound desc = could not find container \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": container with ID starting with c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f not found: ID does not exist" Jul 17 13:21:48.942405 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942354 2560 scope.go:117] "RemoveContainer" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.942541 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942526 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075"} err="failed to get container status \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": rpc error: code = NotFound desc = could not find container \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": container with ID starting with b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075 not found: ID does not exist" Jul 17 13:21:48.942586 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942541 2560 scope.go:117] "RemoveContainer" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.942740 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942724 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} err="failed to get container status \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": rpc error: code = NotFound desc = could not find container \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": container with ID starting with b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb not found: ID does not exist" Jul 17 13:21:48.942787 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942740 2560 scope.go:117] "RemoveContainer" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.942972 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942903 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} err="failed to get container status \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": rpc error: code = NotFound desc = could not find container \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": container with ID starting with f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3 not found: ID does not exist" Jul 17 13:21:48.942972 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.942943 2560 scope.go:117] "RemoveContainer" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.943255 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.943228 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} err="failed to get container status \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": rpc error: code = NotFound desc = could not find container \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": container with ID starting with 1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2 not found: ID does not exist" Jul 17 13:21:48.943330 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.943256 2560 scope.go:117] "RemoveContainer" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.943737 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.943686 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} err="failed to get container status \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": rpc error: code = NotFound desc = could not find container \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": container with ID starting with 053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939 not found: ID does not exist" Jul 17 13:21:48.943737 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.943712 2560 scope.go:117] "RemoveContainer" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.944057 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944026 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} err="failed to get container status \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": rpc error: code = NotFound desc = could not find container \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": container with ID starting with 936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd not found: ID does not exist" Jul 17 13:21:48.944144 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944058 2560 scope.go:117] "RemoveContainer" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.944324 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944302 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} err="failed to get container status \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": rpc error: code = NotFound desc = could not find container \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": container with ID starting with c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f not found: ID does not exist" Jul 17 13:21:48.944324 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944325 2560 scope.go:117] "RemoveContainer" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.944558 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944542 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075"} err="failed to get container status \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": rpc error: code = NotFound desc = could not find container \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": container with ID starting with b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075 not found: ID does not exist" Jul 17 13:21:48.944558 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944557 2560 scope.go:117] "RemoveContainer" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.944788 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944771 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} err="failed to get container status \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": rpc error: code = NotFound desc = could not find container \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": container with ID starting with b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb not found: ID does not exist" Jul 17 13:21:48.944887 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.944789 2560 scope.go:117] "RemoveContainer" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.945046 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945030 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} err="failed to get container status \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": rpc error: code = NotFound desc = could not find container \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": container with ID starting with f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3 not found: ID does not exist" Jul 17 13:21:48.945110 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945047 2560 scope.go:117] "RemoveContainer" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.945294 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945276 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} err="failed to get container status \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": rpc error: code = NotFound desc = could not find container \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": container with ID starting with 1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2 not found: ID does not exist" Jul 17 13:21:48.945345 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945302 2560 scope.go:117] "RemoveContainer" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.945392 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945282 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:21:48.945558 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945538 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} err="failed to get container status \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": rpc error: code = NotFound desc = could not find container \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": container with ID starting with 053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939 not found: ID does not exist" Jul 17 13:21:48.945600 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945570 2560 scope.go:117] "RemoveContainer" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.945743 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945729 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="init-config-reloader" Jul 17 13:21:48.945792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945745 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="init-config-reloader" Jul 17 13:21:48.945792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945753 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy" Jul 17 13:21:48.945792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945759 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy" Jul 17 13:21:48.945792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945774 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="config-reloader" Jul 17 13:21:48.945792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945780 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="config-reloader" Jul 17 13:21:48.945792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945782 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} err="failed to get container status \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": rpc error: code = NotFound desc = could not find container \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": container with ID starting with 936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd not found: ID does not exist" Jul 17 13:21:48.945792 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945793 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="prometheus" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945797 2560 scope.go:117] "RemoveContainer" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945801 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="prometheus" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945810 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="thanos-sidecar" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945819 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="thanos-sidecar" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945828 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-thanos" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945835 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-thanos" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945846 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-web" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945851 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-web" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945938 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-web" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945949 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="config-reloader" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945960 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945971 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="thanos-sidecar" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945981 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="kube-rbac-proxy-thanos" Jul 17 13:21:48.946007 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.945991 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" containerName="prometheus" Jul 17 13:21:48.946386 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946020 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} err="failed to get container status \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": rpc error: code = NotFound desc = could not find container \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": container with ID starting with c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f not found: ID does not exist" Jul 17 13:21:48.946386 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946036 2560 scope.go:117] "RemoveContainer" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.946386 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946229 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075"} err="failed to get container status \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": rpc error: code = NotFound desc = could not find container \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": container with ID starting with b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075 not found: ID does not exist" Jul 17 13:21:48.946386 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946247 2560 scope.go:117] "RemoveContainer" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.946501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946447 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} err="failed to get container status \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": rpc error: code = NotFound desc = could not find container \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": container with ID starting with b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb not found: ID does not exist" Jul 17 13:21:48.946501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946474 2560 scope.go:117] "RemoveContainer" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.946722 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946705 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} err="failed to get container status \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": rpc error: code = NotFound desc = could not find container \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": container with ID starting with f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3 not found: ID does not exist" Jul 17 13:21:48.946722 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946721 2560 scope.go:117] "RemoveContainer" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.946934 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946897 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} err="failed to get container status \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": rpc error: code = NotFound desc = could not find container \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": container with ID starting with 1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2 not found: ID does not exist" Jul 17 13:21:48.946934 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.946932 2560 scope.go:117] "RemoveContainer" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.947163 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947147 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} err="failed to get container status \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": rpc error: code = NotFound desc = could not find container \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": container with ID starting with 053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939 not found: ID does not exist" Jul 17 13:21:48.947250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947163 2560 scope.go:117] "RemoveContainer" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.947361 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947346 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} err="failed to get container status \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": rpc error: code = NotFound desc = could not find container \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": container with ID starting with 936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd not found: ID does not exist" Jul 17 13:21:48.947402 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947363 2560 scope.go:117] "RemoveContainer" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.947593 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947574 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} err="failed to get container status \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": rpc error: code = NotFound desc = could not find container \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": container with ID starting with c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f not found: ID does not exist" Jul 17 13:21:48.947651 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947595 2560 scope.go:117] "RemoveContainer" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.947843 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947819 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075"} err="failed to get container status \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": rpc error: code = NotFound desc = could not find container \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": container with ID starting with b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075 not found: ID does not exist" Jul 17 13:21:48.947843 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.947842 2560 scope.go:117] "RemoveContainer" containerID="b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb" Jul 17 13:21:48.948111 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948094 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb"} err="failed to get container status \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": rpc error: code = NotFound desc = could not find container \"b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb\": container with ID starting with b1e26721c77e231a2fa4ef556754f0c0c8b692e77304836380dbc343e0e63ffb not found: ID does not exist" Jul 17 13:21:48.948236 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948111 2560 scope.go:117] "RemoveContainer" containerID="f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3" Jul 17 13:21:48.948321 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948303 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3"} err="failed to get container status \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": rpc error: code = NotFound desc = could not find container \"f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3\": container with ID starting with f863671ead7259e50da87874700319013f9b0f0c48157c918e3b7afd08b0dba3 not found: ID does not exist" Jul 17 13:21:48.948380 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948323 2560 scope.go:117] "RemoveContainer" containerID="1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2" Jul 17 13:21:48.948519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948502 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2"} err="failed to get container status \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": rpc error: code = NotFound desc = could not find container \"1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2\": container with ID starting with 1e5be31793dd5a815458283b856df21ab171ef9b8473e2ecc60ea0e792017da2 not found: ID does not exist" Jul 17 13:21:48.948519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948518 2560 scope.go:117] "RemoveContainer" containerID="053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939" Jul 17 13:21:48.948860 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948832 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939"} err="failed to get container status \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": rpc error: code = NotFound desc = could not find container \"053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939\": container with ID starting with 053af9c070f78114857efc068a20e570d242ba76f77355c75e9634245419d939 not found: ID does not exist" Jul 17 13:21:48.948974 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.948861 2560 scope.go:117] "RemoveContainer" containerID="936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd" Jul 17 13:21:48.949194 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.949175 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd"} err="failed to get container status \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": rpc error: code = NotFound desc = could not find container \"936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd\": container with ID starting with 936b7702b075dbb3005037a5aa3dee390d67847207477a9bfa7b99da5ed0e3fd not found: ID does not exist" Jul 17 13:21:48.949194 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.949193 2560 scope.go:117] "RemoveContainer" containerID="c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f" Jul 17 13:21:48.949438 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.949415 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f"} err="failed to get container status \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": rpc error: code = NotFound desc = could not find container \"c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f\": container with ID starting with c6a2f89952758a471f0f2f631946be5a6fc1e37385b4350bb88d590c9b62429f not found: ID does not exist" Jul 17 13:21:48.949535 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.949439 2560 scope.go:117] "RemoveContainer" containerID="b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075" Jul 17 13:21:48.949669 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.949650 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075"} err="failed to get container status \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": rpc error: code = NotFound desc = could not find container \"b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075\": container with ID starting with b97eb6c99cd26f2f77f9dd974a2464262a0c37beb297e1028b354998fdd35075 not found: ID does not exist" Jul 17 13:21:48.951094 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.951081 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:48.953629 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.953588 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-gzqtq\"" Jul 17 13:21:48.953629 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.953594 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 17 13:21:48.953795 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.953640 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 17 13:21:48.953795 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.953744 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 17 13:21:48.954053 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954034 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 17 13:21:48.954185 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954068 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 17 13:21:48.954185 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954110 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 17 13:21:48.954185 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954068 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 17 13:21:48.954455 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954380 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 17 13:21:48.954524 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954492 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 17 13:21:48.954524 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954411 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 17 13:21:48.954619 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954581 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-45dck21tsj2ek\"" Jul 17 13:21:48.954619 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954429 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 17 13:21:48.954619 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.954429 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 17 13:21:48.957759 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.957732 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 17 13:21:48.960954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.960939 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 17 13:21:48.963991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:48.963971 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:21:49.074434 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074409 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074434 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074457 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074474 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074489 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074540 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-config-out\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074606 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074578 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kd8v\" (UniqueName: \"kubernetes.io/projected/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-kube-api-access-6kd8v\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074636 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074655 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-config\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-web-config\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074695 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074778 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.074827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.075085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.075085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074855 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.075085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.075085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074894 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.075085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.074922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.100769 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.100740 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c29c4247-b3b1-48e9-b4aa-2eeb44a2b860" path="/var/lib/kubelet/pods/c29c4247-b3b1-48e9-b4aa-2eeb44a2b860/volumes" Jul 17 13:21:49.175691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.175668 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.175808 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.175698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.175808 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.175725 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.175808 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.175758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.175986 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.175860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.176170 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176138 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.176242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176207 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.176293 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.176345 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176312 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176687 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176770 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176887 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.176992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-config-out\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.177037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kd8v\" (UniqueName: \"kubernetes.io/projected/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-kube-api-access-6kd8v\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.177097 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.177157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.177201 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-config\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.177238 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-web-config\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.177264 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.177610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.177285 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.178287 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.178000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.180290 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.179986 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.180290 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.180245 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-config-out\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.180572 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.180546 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.180819 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.180793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.181020 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.180983 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.181386 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.181363 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.181944 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.181813 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.182461 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.182376 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.182461 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.182414 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.183886 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.183417 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.185823 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.184034 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.186620 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.186596 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-web-config\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.186715 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.186664 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.186776 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.186729 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.186859 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.186837 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-config\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.187641 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.187623 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kd8v\" (UniqueName: \"kubernetes.io/projected/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-kube-api-access-6kd8v\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.187694 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.187658 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cf1f11cb-a084-4d66-bbaf-ef3d52061ded-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cf1f11cb-a084-4d66-bbaf-ef3d52061ded\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.261953 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.261932 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:49.397553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.397531 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 13:21:49.399767 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:21:49.399742 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf1f11cb_a084_4d66_bbaf_ef3d52061ded.slice/crio-72aa4327f91716d993e5af70fc5ffd0b1c7e16fbf5cd776e0200ccf362ab843c WatchSource:0}: Error finding container 72aa4327f91716d993e5af70fc5ffd0b1c7e16fbf5cd776e0200ccf362ab843c: Status 404 returned error can't find the container with id 72aa4327f91716d993e5af70fc5ffd0b1c7e16fbf5cd776e0200ccf362ab843c Jul 17 13:21:49.895765 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.895674 2560 generic.go:356] "Generic (PLEG): container finished" podID="cf1f11cb-a084-4d66-bbaf-ef3d52061ded" containerID="324da06a775cd91ff1f477201e960aa52402183ced70c507e63258425a72b8b8" exitCode=0 Jul 17 13:21:49.895765 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.895751 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerDied","Data":"324da06a775cd91ff1f477201e960aa52402183ced70c507e63258425a72b8b8"} Jul 17 13:21:49.896273 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:49.895822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerStarted","Data":"72aa4327f91716d993e5af70fc5ffd0b1c7e16fbf5cd776e0200ccf362ab843c"} Jul 17 13:21:50.903805 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:50.903773 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerStarted","Data":"e3a407fbbb5dbda3c148c99b484974695eeb01a3bbd08f0baa3354b4b3744e3e"} Jul 17 13:21:50.904175 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:50.903837 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerStarted","Data":"d8016501a9c790c44502ab78002e93bb1738effe28e868af31d81e4558c5b11c"} Jul 17 13:21:50.904175 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:50.903863 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerStarted","Data":"03265a649e84aa730969abbe1da2380b26ac9e4017b800e1cd3941c5fd18849b"} Jul 17 13:21:50.904175 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:50.903888 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerStarted","Data":"f4843a328a2d615c11db860957078f20fb6148c165d93a126a684d2316ae936f"} Jul 17 13:21:50.904175 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:50.903931 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerStarted","Data":"d63a2dd38846252028c39e3690a6be6579291d592d8074a780862458c5e3b605"} Jul 17 13:21:50.904175 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:50.903957 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cf1f11cb-a084-4d66-bbaf-ef3d52061ded","Type":"ContainerStarted","Data":"4428fc8225b730a87839882199ba5f5678af3518f602207649a8644e4a6c2a5d"} Jul 17 13:21:50.931647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:50.931593 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.912977539 podStartE2EDuration="2.931571914s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.594375ms" isStatefulPod=false podCreationTimestamp="2026-07-17 13:21:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:21:50.929805795 +0000 UTC m=+153.007919460" watchObservedRunningTime="2026-07-17 13:21:50.931571914 +0000 UTC m=+153.009685579" Jul 17 13:21:51.732112 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:51.732072 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:51.784979 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:51.784945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:54.262475 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:54.262437 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:21:54.276877 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:54.276849 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:54.333493 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:54.333464 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:57.719564 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:57.719524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:21:57.819951 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:21:57.819897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:22:01.221653 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:01.221613 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:22:04.941795 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:04.941753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:22:05.045534 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:05.045495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:22:49.263014 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:49.262985 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:22:49.278225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:49.278200 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:22:50.152284 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:50.152256 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 13:22:59.778815 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:59.778781 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:22:59.830505 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:22:59.830474 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:23:06.124687 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:23:06.124654 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:23:52.402753 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:23:52.402675 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:23:52.504431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:23:52.504399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:23:52.986284 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:23:52.986241 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:18.301279 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:18.301248 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 13:24:29.215044 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.215006 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:29.313950 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.313905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:29.525883 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.525808 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:29.611164 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.611123 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:29.616031 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.616005 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq"] Jul 17 13:24:29.619828 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.619810 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.622339 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.622320 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 17 13:24:29.622450 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.622343 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\"" Jul 17 13:24:29.622647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.622632 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\"" Jul 17 13:24:29.623669 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.623646 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\"" Jul 17 13:24:29.623669 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.623660 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 17 13:24:29.623669 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.623666 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\"" Jul 17 13:24:29.623869 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.623652 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\"" Jul 17 13:24:29.623869 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.623722 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8pmkz\"" Jul 17 13:24:29.634977 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.631997 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq"] Jul 17 13:24:29.672870 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.672844 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg222\" (UniqueName: \"kubernetes.io/projected/20e5f4e8-a880-40df-be70-fd11bea54ae9-kube-api-access-sg222\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.672989 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.672877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.672989 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.672924 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-ui-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.673069 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.672993 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.673069 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.673021 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20e5f4e8-a880-40df-be70-fd11bea54ae9-feast-data\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.673069 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.673040 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-offline-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.722926 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.722858 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:29.773489 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.773455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg222\" (UniqueName: \"kubernetes.io/projected/20e5f4e8-a880-40df-be70-fd11bea54ae9-kube-api-access-sg222\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.773657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.773500 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.773657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.773548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-ui-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.773657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.773582 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.773657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.773605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20e5f4e8-a880-40df-be70-fd11bea54ae9-feast-data\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.773657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.773629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-offline-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.773932 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:24:29.773663 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 17 13:24:29.773932 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:24:29.773686 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found Jul 17 13:24:29.773932 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:24:29.773723 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls podName:20e5f4e8-a880-40df-be70-fd11bea54ae9 nodeName:}" failed. No retries permitted until 2026-07-17 13:24:30.273704889 +0000 UTC m=+312.351818534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls") pod "feast-simple-feast-setup-849fccff88-sq5nq" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9") : secret "feast-simple-feast-setup-online-tls" not found Jul 17 13:24:29.773932 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:24:29.773749 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls podName:20e5f4e8-a880-40df-be70-fd11bea54ae9 nodeName:}" failed. No retries permitted until 2026-07-17 13:24:30.273733668 +0000 UTC m=+312.351847317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls") pod "feast-simple-feast-setup-849fccff88-sq5nq" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9") : secret "feast-simple-feast-setup-registry-tls" not found Jul 17 13:24:29.774142 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.774016 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20e5f4e8-a880-40df-be70-fd11bea54ae9-feast-data\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.776070 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.776015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-ui-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.776070 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.776048 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-offline-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.781707 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.781685 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg222\" (UniqueName: \"kubernetes.io/projected/20e5f4e8-a880-40df-be70-fd11bea54ae9-kube-api-access-sg222\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:29.957944 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:29.957886 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:30.008828 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.008790 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:24:30.277274 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.277245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:30.277645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.277295 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:30.279505 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.279479 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:30.279648 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.279626 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls\") pod \"feast-simple-feast-setup-849fccff88-sq5nq\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:30.543198 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.543128 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:24:30.672176 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.672150 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq"] Jul 17 13:24:30.672733 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:24:30.672712 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20e5f4e8_a880_40df_be70_fd11bea54ae9.slice/crio-e67008c7fb490d0804e3c14cdc6316b9e690401c814d1d131877525e885d3610 WatchSource:0}: Error finding container e67008c7fb490d0804e3c14cdc6316b9e690401c814d1d131877525e885d3610: Status 404 returned error can't find the container with id e67008c7fb490d0804e3c14cdc6316b9e690401c814d1d131877525e885d3610 Jul 17 13:24:30.674680 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:30.674664 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 13:24:31.530751 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:31.530707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerStarted","Data":"e67008c7fb490d0804e3c14cdc6316b9e690401c814d1d131877525e885d3610"} Jul 17 13:24:59.659242 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:24:59.659153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerStarted","Data":"0fa014561a5ae17d05af7958101209d9f9b57b20d78eaee6c11b04709042dbef"} Jul 17 13:25:04.680857 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:04.680826 2560 generic.go:356] "Generic (PLEG): container finished" podID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerID="0fa014561a5ae17d05af7958101209d9f9b57b20d78eaee6c11b04709042dbef" exitCode=0 Jul 17 13:25:04.681337 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:04.680901 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerDied","Data":"0fa014561a5ae17d05af7958101209d9f9b57b20d78eaee6c11b04709042dbef"} Jul 17 13:25:05.687508 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:05.687466 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerStarted","Data":"80ce7f7d9d9d1f21746c9edf194bacb8bd594e9a9acccc7441922a7e14d8b42a"} Jul 17 13:25:11.715873 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:11.715842 2560 generic.go:356] "Generic (PLEG): container finished" podID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerID="80ce7f7d9d9d1f21746c9edf194bacb8bd594e9a9acccc7441922a7e14d8b42a" exitCode=0 Jul 17 13:25:11.716282 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:11.715903 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerDied","Data":"80ce7f7d9d9d1f21746c9edf194bacb8bd594e9a9acccc7441922a7e14d8b42a"} Jul 17 13:25:12.726945 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:12.726887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerStarted","Data":"f9d9ee0292dcc0f4af8317f611805bb39dd2fe4965b68aba52faf20369ffbeb8"} Jul 17 13:25:12.727422 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:12.726965 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerStarted","Data":"58e025efb81169c3dbb8fb966ed5acf9bdba6539f9e7e65a6f711592d817d8a5"} Jul 17 13:25:12.727422 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:12.726990 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerStarted","Data":"deb010ea0f9c6e1b46bfba3413a521120b6207c229054ee725e26c1d17a86692"} Jul 17 13:25:12.727422 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:12.727009 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerStarted","Data":"1ebad3a6b3391e8daf56057c71ad8e5df0f9b7ac719c85b35afcb502f3871c2d"} Jul 17 13:25:12.752780 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:12.752710 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podStartSLOduration=15.226634759 podStartE2EDuration="43.752687439s" totalImagesPullingTime="28.52605268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:25:12.749730653 +0000 UTC m=+354.827844319" watchObservedRunningTime="2026-07-17 13:25:12.752687439 +0000 UTC m=+354.830801104" Jul 17 13:25:15.544093 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.544051 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:15.544661 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.544642 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:15.544793 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.544781 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:15.545164 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.545148 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:15.546353 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.546304 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jul 17 13:25:15.546353 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.546341 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused" Jul 17 13:25:15.546526 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.546317 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused" Jul 17 13:25:15.546735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:15.546708 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jul 17 13:25:18.545258 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.545228 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.545691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.545368 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.545691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.545584 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.546040 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.546016 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.546192 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.546059 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.546192 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.546078 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.546192 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.546138 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.546441 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.546419 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.546512 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.546478 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.550664 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.550644 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.761048 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.761019 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:18.764150 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:18.764129 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:25:19.557036 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:19.557006 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:25:38.279431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:38.279394 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:25:38.381436 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:38.381399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:25:38.729497 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:38.729450 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:25:38.780140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:38.780112 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:25:48.761801 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:48.761753 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:25:48.812604 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:48.812570 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:25:48.869502 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:25:48.869470 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:03.119465 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:03.119425 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:03.171078 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:03.171045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:05.068163 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:05.068129 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:05.123932 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:05.123866 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:09.796565 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:09.796526 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:09.855172 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:09.855137 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:20.979036 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:20.978975 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:20.980720 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:20.980688 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq"] Jul 17 13:26:20.981010 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:20.980952 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="registry" containerID="cri-o://1ebad3a6b3391e8daf56057c71ad8e5df0f9b7ac719c85b35afcb502f3871c2d" gracePeriod=30 Jul 17 13:26:20.981119 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:20.981048 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" containerID="cri-o://deb010ea0f9c6e1b46bfba3413a521120b6207c229054ee725e26c1d17a86692" gracePeriod=30 Jul 17 13:26:20.981185 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:20.981048 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" containerID="cri-o://f9d9ee0292dcc0f4af8317f611805bb39dd2fe4965b68aba52faf20369ffbeb8" gracePeriod=30 Jul 17 13:26:20.981250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:20.981231 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="offline" containerID="cri-o://58e025efb81169c3dbb8fb966ed5acf9bdba6539f9e7e65a6f711592d817d8a5" gracePeriod=30 Jul 17 13:26:21.077967 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.077935 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:21.179817 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.179784 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:21.280837 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.280746 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:21.340705 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.340670 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk"] Jul 17 13:26:21.345497 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.345472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.348002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.347978 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-lm4v2\"" Jul 17 13:26:21.355258 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.355228 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk"] Jul 17 13:26:21.424569 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.424529 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:21.456169 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.456146 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-offline-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.456263 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.456180 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.456263 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.456211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4snm\" (UniqueName: \"kubernetes.io/projected/e0d8909f-05bf-48ae-ace8-ac8704e91784-kube-api-access-w4snm\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.456335 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.456275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-registry-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.456335 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.456292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-ui-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.456335 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.456327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d8909f-05bf-48ae-ace8-ac8704e91784-feast-data\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.525452 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.525400 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:21.557544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.557468 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-registry-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.557544 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.557503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-ui-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.557737 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.557556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d8909f-05bf-48ae-ace8-ac8704e91784-feast-data\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.557737 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.557622 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-offline-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.557848 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.557741 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.557848 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.557797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4snm\" (UniqueName: \"kubernetes.io/projected/e0d8909f-05bf-48ae-ace8-ac8704e91784-kube-api-access-w4snm\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.558084 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:26:21.558042 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found Jul 17 13:26:21.558216 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.558047 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d8909f-05bf-48ae-ace8-ac8704e91784-feast-data\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.558282 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:26:21.558217 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls podName:e0d8909f-05bf-48ae-ace8-ac8704e91784 nodeName:}" failed. No retries permitted until 2026-07-17 13:26:22.058142495 +0000 UTC m=+424.136256137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls") pod "feast-simple-feast-setup-849fccff88-x9gtk" (UID: "e0d8909f-05bf-48ae-ace8-ac8704e91784") : secret "feast-simple-feast-setup-online-tls" not found Jul 17 13:26:21.560083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.560055 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-registry-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.560547 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.560525 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-offline-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.560614 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.560535 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-ui-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.565938 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.565896 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4snm\" (UniqueName: \"kubernetes.io/projected/e0d8909f-05bf-48ae-ace8-ac8704e91784-kube-api-access-w4snm\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:21.624508 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.624475 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:21.726113 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:21.726079 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:22.018530 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.018501 2560 generic.go:356] "Generic (PLEG): container finished" podID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerID="f9d9ee0292dcc0f4af8317f611805bb39dd2fe4965b68aba52faf20369ffbeb8" exitCode=0 Jul 17 13:26:22.018530 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.018525 2560 generic.go:356] "Generic (PLEG): container finished" podID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerID="deb010ea0f9c6e1b46bfba3413a521120b6207c229054ee725e26c1d17a86692" exitCode=0 Jul 17 13:26:22.018940 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.018543 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerDied","Data":"f9d9ee0292dcc0f4af8317f611805bb39dd2fe4965b68aba52faf20369ffbeb8"} Jul 17 13:26:22.018940 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.018572 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerDied","Data":"deb010ea0f9c6e1b46bfba3413a521120b6207c229054ee725e26c1d17a86692"} Jul 17 13:26:22.061903 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.061873 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:22.064171 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.064152 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls\") pod \"feast-simple-feast-setup-849fccff88-x9gtk\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:22.255793 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.255759 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:22.385884 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:22.385854 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk"] Jul 17 13:26:22.386381 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:26:22.386343 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0d8909f_05bf_48ae_ace8_ac8704e91784.slice/crio-49f37a73d087e4800ffaf9cd96af08e343e81e26acdf0248bcf4c09c68367cf2 WatchSource:0}: Error finding container 49f37a73d087e4800ffaf9cd96af08e343e81e26acdf0248bcf4c09c68367cf2: Status 404 returned error can't find the container with id 49f37a73d087e4800ffaf9cd96af08e343e81e26acdf0248bcf4c09c68367cf2 Jul 17 13:26:23.023932 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:23.023876 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerStarted","Data":"3360843bf8c1d07643e930a9b55c4844033b41dbba19b39f128acf289f216320"} Jul 17 13:26:23.023932 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:23.023934 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerStarted","Data":"49f37a73d087e4800ffaf9cd96af08e343e81e26acdf0248bcf4c09c68367cf2"} Jul 17 13:26:28.046488 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:28.046456 2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerID="3360843bf8c1d07643e930a9b55c4844033b41dbba19b39f128acf289f216320" exitCode=0 Jul 17 13:26:28.046955 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:28.046534 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerDied","Data":"3360843bf8c1d07643e930a9b55c4844033b41dbba19b39f128acf289f216320"} Jul 17 13:26:28.546541 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:28.546496 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jul 17 13:26:28.761323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:28.761269 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jul 17 13:26:29.052245 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:29.052212 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerStarted","Data":"ff6dd013026ec7ea37abf0b50e6731bcf6303c86cbca30f5983cac03df38a97a"} Jul 17 13:26:34.073981 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:34.073940 2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerID="ff6dd013026ec7ea37abf0b50e6731bcf6303c86cbca30f5983cac03df38a97a" exitCode=0 Jul 17 13:26:34.074351 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:34.074003 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerDied","Data":"ff6dd013026ec7ea37abf0b50e6731bcf6303c86cbca30f5983cac03df38a97a"} Jul 17 13:26:35.084141 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:35.084099 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerStarted","Data":"dc196c7d7a2bb246fc516d51461ea937221e880094a608e54e7b7454ed280ce3"} Jul 17 13:26:35.084664 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:35.084161 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerStarted","Data":"a6bb4aff52593b5791bfba189e4b55b2aa6e92c35bee8b1c020aae224bbad519"} Jul 17 13:26:35.084664 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:35.084181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerStarted","Data":"cf049b700b61b6df0e5f38dee41c78d4cf37030de8433d870837b1103863a474"} Jul 17 13:26:35.084664 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:35.084199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerStarted","Data":"3dfa2636ded0709afa9a2d297aa513294a8399bcb4a9ba864b4461444979f81a"} Jul 17 13:26:35.111798 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:35.111738 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podStartSLOduration=14.11171592 podStartE2EDuration="14.11171592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:26:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:26:35.108620857 +0000 UTC m=+437.186734524" watchObservedRunningTime="2026-07-17 13:26:35.11171592 +0000 UTC m=+437.189829585" Jul 17 13:26:37.256776 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.256736 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:37.257292 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.256802 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:37.257292 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.256822 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:37.257292 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.256839 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:37.259002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.258959 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused" Jul 17 13:26:37.259002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.258982 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jul 17 13:26:37.259202 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.258981 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused" Jul 17 13:26:37.259202 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:37.259047 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jul 17 13:26:38.546949 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:38.546878 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jul 17 13:26:38.762204 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:38.762159 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jul 17 13:26:40.256837 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.256796 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jul 17 13:26:40.257494 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.257466 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:40.257788 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.257762 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:40.258140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.258118 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:40.258253 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.258158 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:40.258377 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.258362 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:40.258506 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.258420 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:40.263203 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:40.263182 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:41.121904 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:41.120344 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:41.125177 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:41.125148 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:42.256767 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:42.256741 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:43.256546 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:43.256514 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:44.131043 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:44.131016 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:26:44.350587 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:44.350549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:26:48.546037 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:48.545992 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused" Jul 17 13:26:48.546504 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:48.546157 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:26:48.761645 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:48.761604 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused" Jul 17 13:26:48.761832 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:48.761731 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:26:51.163257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.163225 2560 generic.go:356] "Generic (PLEG): container finished" podID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerID="58e025efb81169c3dbb8fb966ed5acf9bdba6539f9e7e65a6f711592d817d8a5" exitCode=137 Jul 17 13:26:51.163257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.163249 2560 generic.go:356] "Generic (PLEG): container finished" podID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerID="1ebad3a6b3391e8daf56057c71ad8e5df0f9b7ac719c85b35afcb502f3871c2d" exitCode=137 Jul 17 13:26:51.163621 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.163294 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerDied","Data":"58e025efb81169c3dbb8fb966ed5acf9bdba6539f9e7e65a6f711592d817d8a5"} Jul 17 13:26:51.163621 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.163332 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerDied","Data":"1ebad3a6b3391e8daf56057c71ad8e5df0f9b7ac719c85b35afcb502f3871c2d"} Jul 17 13:26:51.621318 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.621293 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:26:51.732235 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.732206 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls\") pod \"20e5f4e8-a880-40df-be70-fd11bea54ae9\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " Jul 17 13:26:51.732379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.732253 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20e5f4e8-a880-40df-be70-fd11bea54ae9-feast-data\") pod \"20e5f4e8-a880-40df-be70-fd11bea54ae9\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " Jul 17 13:26:51.732379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.732277 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-offline-tls\") pod \"20e5f4e8-a880-40df-be70-fd11bea54ae9\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " Jul 17 13:26:51.732379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.732321 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls\") pod \"20e5f4e8-a880-40df-be70-fd11bea54ae9\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " Jul 17 13:26:51.732379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.732337 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-ui-tls\") pod \"20e5f4e8-a880-40df-be70-fd11bea54ae9\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " Jul 17 13:26:51.732379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.732369 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg222\" (UniqueName: \"kubernetes.io/projected/20e5f4e8-a880-40df-be70-fd11bea54ae9-kube-api-access-sg222\") pod \"20e5f4e8-a880-40df-be70-fd11bea54ae9\" (UID: \"20e5f4e8-a880-40df-be70-fd11bea54ae9\") " Jul 17 13:26:51.732802 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.732762 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20e5f4e8-a880-40df-be70-fd11bea54ae9-feast-data" (OuterVolumeSpecName: "feast-data") pod "20e5f4e8-a880-40df-be70-fd11bea54ae9" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:26:51.734616 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.734594 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "20e5f4e8-a880-40df-be70-fd11bea54ae9" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:26:51.734941 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.734892 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "20e5f4e8-a880-40df-be70-fd11bea54ae9" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:26:51.735039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.734903 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "20e5f4e8-a880-40df-be70-fd11bea54ae9" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:26:51.735039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.734953 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20e5f4e8-a880-40df-be70-fd11bea54ae9-kube-api-access-sg222" (OuterVolumeSpecName: "kube-api-access-sg222") pod "20e5f4e8-a880-40df-be70-fd11bea54ae9" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9"). InnerVolumeSpecName "kube-api-access-sg222". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:26:51.735039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.734969 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls" (OuterVolumeSpecName: "online-tls") pod "20e5f4e8-a880-40df-be70-fd11bea54ae9" (UID: "20e5f4e8-a880-40df-be70-fd11bea54ae9"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:26:51.833624 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.833562 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-online-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:26:51.833624 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.833585 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-ui-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:26:51.833624 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.833594 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg222\" (UniqueName: \"kubernetes.io/projected/20e5f4e8-a880-40df-be70-fd11bea54ae9-kube-api-access-sg222\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:26:51.833624 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.833604 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-registry-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:26:51.833624 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.833612 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/20e5f4e8-a880-40df-be70-fd11bea54ae9-feast-data\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:26:51.833624 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:51.833619 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/20e5f4e8-a880-40df-be70-fd11bea54ae9-offline-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:26:52.170763 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.170677 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" event={"ID":"20e5f4e8-a880-40df-be70-fd11bea54ae9","Type":"ContainerDied","Data":"e67008c7fb490d0804e3c14cdc6316b9e690401c814d1d131877525e885d3610"} Jul 17 13:26:52.170763 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.170738 2560 scope.go:117] "RemoveContainer" containerID="f9d9ee0292dcc0f4af8317f611805bb39dd2fe4965b68aba52faf20369ffbeb8" Jul 17 13:26:52.171222 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.170740 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq" Jul 17 13:26:52.179943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.179927 2560 scope.go:117] "RemoveContainer" containerID="58e025efb81169c3dbb8fb966ed5acf9bdba6539f9e7e65a6f711592d817d8a5" Jul 17 13:26:52.187379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.187363 2560 scope.go:117] "RemoveContainer" containerID="deb010ea0f9c6e1b46bfba3413a521120b6207c229054ee725e26c1d17a86692" Jul 17 13:26:52.192758 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.192736 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq"] Jul 17 13:26:52.195501 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.195478 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-sq5nq"] Jul 17 13:26:52.195854 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.195834 2560 scope.go:117] "RemoveContainer" containerID="1ebad3a6b3391e8daf56057c71ad8e5df0f9b7ac719c85b35afcb502f3871c2d" Jul 17 13:26:52.202842 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.202824 2560 scope.go:117] "RemoveContainer" containerID="80ce7f7d9d9d1f21746c9edf194bacb8bd594e9a9acccc7441922a7e14d8b42a" Jul 17 13:26:52.209642 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:52.209627 2560 scope.go:117] "RemoveContainer" containerID="0fa014561a5ae17d05af7958101209d9f9b57b20d78eaee6c11b04709042dbef" Jul 17 13:26:53.100566 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:26:53.100533 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" path="/var/lib/kubelet/pods/20e5f4e8-a880-40df-be70-fd11bea54ae9/volumes" Jul 17 13:27:46.721099 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.721034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:27:46.823431 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.823401 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:27:46.871720 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.871687 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9"] Jul 17 13:27:46.872049 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872037 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="feast-init" Jul 17 13:27:46.872100 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872050 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="feast-init" Jul 17 13:27:46.872100 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872058 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="registry" Jul 17 13:27:46.872100 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872065 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="registry" Jul 17 13:27:46.872100 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872072 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="offline" Jul 17 13:27:46.872100 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872079 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="offline" Jul 17 13:27:46.872100 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872088 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" Jul 17 13:27:46.872100 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872093 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872104 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="feast-apply" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872115 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="feast-apply" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872136 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872143 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872206 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="registry" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872215 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="offline" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872221 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="ui" Jul 17 13:27:46.872323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.872228 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="20e5f4e8-a880-40df-be70-fd11bea54ae9" containerName="online" Jul 17 13:27:46.875351 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.875334 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:46.878250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878048 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\"" Jul 17 13:27:46.878250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878119 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\"" Jul 17 13:27:46.878250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878124 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\"" Jul 17 13:27:46.878250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878124 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\"" Jul 17 13:27:46.878250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878127 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\"" Jul 17 13:27:46.878250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878165 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\"" Jul 17 13:27:46.878250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878223 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-gh5kz\"" Jul 17 13:27:46.878615 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.878544 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\"" Jul 17 13:27:46.886781 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.886759 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9"] Jul 17 13:27:46.894676 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.894652 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:27:46.994646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.994570 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bb59cd43-ff2b-42bb-a897-c591e24c1592-feast-data\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:46.994646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.994612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj4qv\" (UniqueName: \"kubernetes.io/projected/bb59cd43-ff2b-42bb-a897-c591e24c1592-kube-api-access-rj4qv\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:46.994646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.994644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-online-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:46.994846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.994717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-ui-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:46.994846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.994753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-offline-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:46.994846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:46.994817 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bb59cd43-ff2b-42bb-a897-c591e24c1592-registry-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.017947 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.017890 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:27:47.096139 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-online-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.096305 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-ui-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.096305 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096196 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-offline-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.096305 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bb59cd43-ff2b-42bb-a897-c591e24c1592-registry-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.096517 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096381 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bb59cd43-ff2b-42bb-a897-c591e24c1592-feast-data\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.096517 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096448 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj4qv\" (UniqueName: \"kubernetes.io/projected/bb59cd43-ff2b-42bb-a897-c591e24c1592-kube-api-access-rj4qv\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.096731 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096711 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bb59cd43-ff2b-42bb-a897-c591e24c1592-feast-data\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.096990 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.096970 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bb59cd43-ff2b-42bb-a897-c591e24c1592-registry-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.098724 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.098698 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-online-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.098927 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.098896 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-offline-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.099119 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.099102 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-ui-tls\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.104506 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.104485 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj4qv\" (UniqueName: \"kubernetes.io/projected/bb59cd43-ff2b-42bb-a897-c591e24c1592-kube-api-access-rj4qv\") pod \"feast-simple-feast-remote-setup-755586bd9-hl2f9\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.116188 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.116167 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:27:47.184587 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.184556 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:27:47.321109 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.321085 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9"] Jul 17 13:27:47.323000 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:27:47.322967 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb59cd43_ff2b_42bb_a897_c591e24c1592.slice/crio-ec551ee68af5d50e0efe1cffb18359e36ba92d47000b10148082aae4d0f585ba WatchSource:0}: Error finding container ec551ee68af5d50e0efe1cffb18359e36ba92d47000b10148082aae4d0f585ba: Status 404 returned error can't find the container with id ec551ee68af5d50e0efe1cffb18359e36ba92d47000b10148082aae4d0f585ba Jul 17 13:27:47.405301 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.405275 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerStarted","Data":"2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3"} Jul 17 13:27:47.405435 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:47.405321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerStarted","Data":"ec551ee68af5d50e0efe1cffb18359e36ba92d47000b10148082aae4d0f585ba"} Jul 17 13:27:52.428209 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:52.428175 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerID="2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3" exitCode=0 Jul 17 13:27:52.428579 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:52.428248 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerDied","Data":"2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3"} Jul 17 13:27:53.434369 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:53.434325 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerStarted","Data":"e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec"} Jul 17 13:27:58.457540 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:58.457505 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerID="e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec" exitCode=0 Jul 17 13:27:58.457954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:58.457575 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerDied","Data":"e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec"} Jul 17 13:27:59.466167 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:59.466129 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerStarted","Data":"9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352"} Jul 17 13:27:59.466632 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:59.466181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerStarted","Data":"e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb"} Jul 17 13:27:59.466632 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:59.466195 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerStarted","Data":"b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356"} Jul 17 13:27:59.488937 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:27:59.488856 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podStartSLOduration=13.4888342 podStartE2EDuration="13.4888342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:27:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:27:59.486762386 +0000 UTC m=+521.564876051" watchObservedRunningTime="2026-07-17 13:27:59.4888342 +0000 UTC m=+521.566947866" Jul 17 13:28:02.185067 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:02.185024 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:02.185560 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:02.185090 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:02.185560 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:02.185110 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:02.186580 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:02.186545 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused" Jul 17 13:28:02.186734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:02.186571 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 17 13:28:02.186734 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:02.186549 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jul 17 13:28:05.186178 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.186140 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.186611 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.186597 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.186678 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.186633 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.186946 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.186925 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.193039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.193016 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.499138 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.499051 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.499138 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.499097 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.499414 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.499392 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.503614 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.503593 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:28:05.725464 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:28:05.725422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:02.712608 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:02.712567 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:08.221299 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.221256 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:08.222455 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.222427 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9"] Jul 17 13:29:08.222736 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.222693 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" containerID="cri-o://b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356" gracePeriod=30 Jul 17 13:29:08.222846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.222717 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" containerID="cri-o://9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352" gracePeriod=30 Jul 17 13:29:08.222846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.222742 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="offline" containerID="cri-o://e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb" gracePeriod=30 Jul 17 13:29:08.274416 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.274381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:08.332039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.332008 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk"] Jul 17 13:29:08.332301 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.332260 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="registry" containerID="cri-o://3dfa2636ded0709afa9a2d297aa513294a8399bcb4a9ba864b4461444979f81a" gracePeriod=30 Jul 17 13:29:08.332517 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.332316 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="offline" containerID="cri-o://a6bb4aff52593b5791bfba189e4b55b2aa6e92c35bee8b1c020aae224bbad519" gracePeriod=30 Jul 17 13:29:08.332643 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.332327 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" containerID="cri-o://cf049b700b61b6df0e5f38dee41c78d4cf37030de8433d870837b1103863a474" gracePeriod=30 Jul 17 13:29:08.332709 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.332374 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" containerID="cri-o://dc196c7d7a2bb246fc516d51461ea937221e880094a608e54e7b7454ed280ce3" gracePeriod=30 Jul 17 13:29:08.338454 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.338423 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:08.418069 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.418034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:08.518982 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.518872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:08.767220 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.767186 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerID="9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352" exitCode=0 Jul 17 13:29:08.767384 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:08.767231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerDied","Data":"9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352"} Jul 17 13:29:09.774344 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:09.774315 2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerID="dc196c7d7a2bb246fc516d51461ea937221e880094a608e54e7b7454ed280ce3" exitCode=0 Jul 17 13:29:09.774344 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:09.774337 2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerID="cf049b700b61b6df0e5f38dee41c78d4cf37030de8433d870837b1103863a474" exitCode=0 Jul 17 13:29:09.774814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:09.774389 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerDied","Data":"dc196c7d7a2bb246fc516d51461ea937221e880094a608e54e7b7454ed280ce3"} Jul 17 13:29:09.774814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:09.774440 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerDied","Data":"cf049b700b61b6df0e5f38dee41c78d4cf37030de8433d870837b1103863a474"} Jul 17 13:29:09.776886 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:09.776866 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerID="b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356" exitCode=0 Jul 17 13:29:09.777022 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:09.776942 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerDied","Data":"b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356"} Jul 17 13:29:11.120225 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:11.120188 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jul 17 13:29:13.515730 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:13.515681 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:14.130918 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:14.130879 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jul 17 13:29:15.186937 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:15.186890 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jul 17 13:29:15.499315 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:15.499226 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 17 13:29:21.120378 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:21.120338 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jul 17 13:29:24.131227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:24.131182 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jul 17 13:29:25.187610 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:25.187568 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jul 17 13:29:25.500247 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:25.500136 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 17 13:29:31.121039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:31.120994 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused" Jul 17 13:29:31.123541 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:31.121117 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:29:34.130620 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:34.130573 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused" Jul 17 13:29:34.131198 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:34.130715 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:29:35.187401 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:35.187366 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused" Jul 17 13:29:35.187763 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:35.187512 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:29:35.499285 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:35.499192 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused" Jul 17 13:29:35.499459 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:35.499323 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:29:38.864322 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.864296 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:29:38.888329 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888298 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-ui-tls\") pod \"bb59cd43-ff2b-42bb-a897-c591e24c1592\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " Jul 17 13:29:38.888539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888360 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bb59cd43-ff2b-42bb-a897-c591e24c1592-registry-tls\") pod \"bb59cd43-ff2b-42bb-a897-c591e24c1592\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " Jul 17 13:29:38.888539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888387 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-online-tls\") pod \"bb59cd43-ff2b-42bb-a897-c591e24c1592\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " Jul 17 13:29:38.888539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888444 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bb59cd43-ff2b-42bb-a897-c591e24c1592-feast-data\") pod \"bb59cd43-ff2b-42bb-a897-c591e24c1592\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " Jul 17 13:29:38.888539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888487 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-offline-tls\") pod \"bb59cd43-ff2b-42bb-a897-c591e24c1592\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " Jul 17 13:29:38.888539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888516 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj4qv\" (UniqueName: \"kubernetes.io/projected/bb59cd43-ff2b-42bb-a897-c591e24c1592-kube-api-access-rj4qv\") pod \"bb59cd43-ff2b-42bb-a897-c591e24c1592\" (UID: \"bb59cd43-ff2b-42bb-a897-c591e24c1592\") " Jul 17 13:29:38.888834 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888747 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb59cd43-ff2b-42bb-a897-c591e24c1592-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bb59cd43-ff2b-42bb-a897-c591e24c1592" (UID: "bb59cd43-ff2b-42bb-a897-c591e24c1592"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 13:29:38.889203 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.888930 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bb59cd43-ff2b-42bb-a897-c591e24c1592-registry-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:38.889203 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.889126 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb59cd43-ff2b-42bb-a897-c591e24c1592-feast-data" (OuterVolumeSpecName: "feast-data") pod "bb59cd43-ff2b-42bb-a897-c591e24c1592" (UID: "bb59cd43-ff2b-42bb-a897-c591e24c1592"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:29:38.891201 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.891175 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb59cd43-ff2b-42bb-a897-c591e24c1592-kube-api-access-rj4qv" (OuterVolumeSpecName: "kube-api-access-rj4qv") pod "bb59cd43-ff2b-42bb-a897-c591e24c1592" (UID: "bb59cd43-ff2b-42bb-a897-c591e24c1592"). InnerVolumeSpecName "kube-api-access-rj4qv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:29:38.891201 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.891186 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "bb59cd43-ff2b-42bb-a897-c591e24c1592" (UID: "bb59cd43-ff2b-42bb-a897-c591e24c1592"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:29:38.891379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.891355 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-online-tls" (OuterVolumeSpecName: "online-tls") pod "bb59cd43-ff2b-42bb-a897-c591e24c1592" (UID: "bb59cd43-ff2b-42bb-a897-c591e24c1592"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:29:38.891469 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.891448 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "bb59cd43-ff2b-42bb-a897-c591e24c1592" (UID: "bb59cd43-ff2b-42bb-a897-c591e24c1592"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:29:38.900601 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.900576 2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerID="a6bb4aff52593b5791bfba189e4b55b2aa6e92c35bee8b1c020aae224bbad519" exitCode=137 Jul 17 13:29:38.900601 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.900596 2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerID="3dfa2636ded0709afa9a2d297aa513294a8399bcb4a9ba864b4461444979f81a" exitCode=137 Jul 17 13:29:38.900793 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.900652 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerDied","Data":"a6bb4aff52593b5791bfba189e4b55b2aa6e92c35bee8b1c020aae224bbad519"} Jul 17 13:29:38.900793 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.900701 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerDied","Data":"3dfa2636ded0709afa9a2d297aa513294a8399bcb4a9ba864b4461444979f81a"} Jul 17 13:29:38.904090 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.904069 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerID="e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb" exitCode=137 Jul 17 13:29:38.904191 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.904138 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerDied","Data":"e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb"} Jul 17 13:29:38.904191 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.904166 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" Jul 17 13:29:38.904191 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.904178 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9" event={"ID":"bb59cd43-ff2b-42bb-a897-c591e24c1592","Type":"ContainerDied","Data":"ec551ee68af5d50e0efe1cffb18359e36ba92d47000b10148082aae4d0f585ba"} Jul 17 13:29:38.904353 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.904203 2560 scope.go:117] "RemoveContainer" containerID="9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352" Jul 17 13:29:38.913877 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.913863 2560 scope.go:117] "RemoveContainer" containerID="e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb" Jul 17 13:29:38.922988 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.922971 2560 scope.go:117] "RemoveContainer" containerID="b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356" Jul 17 13:29:38.928351 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.928320 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9"] Jul 17 13:29:38.932042 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.932024 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-755586bd9-hl2f9"] Jul 17 13:29:38.934318 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.934303 2560 scope.go:117] "RemoveContainer" containerID="e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec" Jul 17 13:29:38.941680 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.941651 2560 scope.go:117] "RemoveContainer" containerID="2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3" Jul 17 13:29:38.950083 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.950063 2560 scope.go:117] "RemoveContainer" containerID="9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352" Jul 17 13:29:38.950346 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:29:38.950327 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352\": container with ID starting with 9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352 not found: ID does not exist" containerID="9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352" Jul 17 13:29:38.950394 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.950355 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352"} err="failed to get container status \"9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352\": rpc error: code = NotFound desc = could not find container \"9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352\": container with ID starting with 9db67a4adba0a9cbbfbd734275cc7af06246c11447b722cb639062de94ba4352 not found: ID does not exist" Jul 17 13:29:38.950394 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.950375 2560 scope.go:117] "RemoveContainer" containerID="e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb" Jul 17 13:29:38.950589 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:29:38.950574 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb\": container with ID starting with e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb not found: ID does not exist" containerID="e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb" Jul 17 13:29:38.950636 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.950594 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb"} err="failed to get container status \"e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb\": rpc error: code = NotFound desc = could not find container \"e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb\": container with ID starting with e0779180eecdefef4df1e33b0388bed98fb5f61dbde22e7d763d5cff69e16dcb not found: ID does not exist" Jul 17 13:29:38.950636 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.950610 2560 scope.go:117] "RemoveContainer" containerID="b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356" Jul 17 13:29:38.950808 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:29:38.950790 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356\": container with ID starting with b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356 not found: ID does not exist" containerID="b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356" Jul 17 13:29:38.950870 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.950815 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356"} err="failed to get container status \"b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356\": rpc error: code = NotFound desc = could not find container \"b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356\": container with ID starting with b9d1f217dba3fc0099908a426729f734abd637b17b0b6fa989c3a1e596628356 not found: ID does not exist" Jul 17 13:29:38.950870 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.950838 2560 scope.go:117] "RemoveContainer" containerID="e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec" Jul 17 13:29:38.951105 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:29:38.951085 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec\": container with ID starting with e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec not found: ID does not exist" containerID="e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec" Jul 17 13:29:38.951204 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.951108 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec"} err="failed to get container status \"e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec\": rpc error: code = NotFound desc = could not find container \"e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec\": container with ID starting with e06966239981c2f8712334689063c423f242b42cdcd200817e46a3a7c43bb6ec not found: ID does not exist" Jul 17 13:29:38.951204 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.951122 2560 scope.go:117] "RemoveContainer" containerID="2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3" Jul 17 13:29:38.951340 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:29:38.951323 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3\": container with ID starting with 2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3 not found: ID does not exist" containerID="2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3" Jul 17 13:29:38.951384 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.951347 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3"} err="failed to get container status \"2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3\": rpc error: code = NotFound desc = could not find container \"2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3\": container with ID starting with 2a7997b538a94a89b3ca2895748263d8fc6521066dfdd75893817f62bd6e73f3 not found: ID does not exist" Jul 17 13:29:38.986143 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.986124 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:29:38.989486 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989430 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls\") pod \"e0d8909f-05bf-48ae-ace8-ac8704e91784\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " Jul 17 13:29:38.989486 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989464 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d8909f-05bf-48ae-ace8-ac8704e91784-feast-data\") pod \"e0d8909f-05bf-48ae-ace8-ac8704e91784\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " Jul 17 13:29:38.989621 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989515 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4snm\" (UniqueName: \"kubernetes.io/projected/e0d8909f-05bf-48ae-ace8-ac8704e91784-kube-api-access-w4snm\") pod \"e0d8909f-05bf-48ae-ace8-ac8704e91784\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " Jul 17 13:29:38.989621 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989565 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-ui-tls\") pod \"e0d8909f-05bf-48ae-ace8-ac8704e91784\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " Jul 17 13:29:38.989724 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989626 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-offline-tls\") pod \"e0d8909f-05bf-48ae-ace8-ac8704e91784\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " Jul 17 13:29:38.989724 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989652 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-registry-tls\") pod \"e0d8909f-05bf-48ae-ace8-ac8704e91784\" (UID: \"e0d8909f-05bf-48ae-ace8-ac8704e91784\") " Jul 17 13:29:38.990140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989952 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bb59cd43-ff2b-42bb-a897-c591e24c1592-feast-data\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:38.990140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989974 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-offline-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:38.990140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.989988 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj4qv\" (UniqueName: \"kubernetes.io/projected/bb59cd43-ff2b-42bb-a897-c591e24c1592-kube-api-access-rj4qv\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:38.990140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.990005 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-ui-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:38.990140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.990018 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bb59cd43-ff2b-42bb-a897-c591e24c1592-online-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:38.990140 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.990081 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0d8909f-05bf-48ae-ace8-ac8704e91784-feast-data" (OuterVolumeSpecName: "feast-data") pod "e0d8909f-05bf-48ae-ace8-ac8704e91784" (UID: "e0d8909f-05bf-48ae-ace8-ac8704e91784"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:29:38.991704 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.991676 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e0d8909f-05bf-48ae-ace8-ac8704e91784" (UID: "e0d8909f-05bf-48ae-ace8-ac8704e91784"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:29:38.991789 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.991717 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e0d8909f-05bf-48ae-ace8-ac8704e91784" (UID: "e0d8909f-05bf-48ae-ace8-ac8704e91784"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:29:38.991789 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.991763 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0d8909f-05bf-48ae-ace8-ac8704e91784-kube-api-access-w4snm" (OuterVolumeSpecName: "kube-api-access-w4snm") pod "e0d8909f-05bf-48ae-ace8-ac8704e91784" (UID: "e0d8909f-05bf-48ae-ace8-ac8704e91784"). InnerVolumeSpecName "kube-api-access-w4snm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:29:38.992014 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.991991 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls" (OuterVolumeSpecName: "online-tls") pod "e0d8909f-05bf-48ae-ace8-ac8704e91784" (UID: "e0d8909f-05bf-48ae-ace8-ac8704e91784"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:29:38.992079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:38.992022 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "e0d8909f-05bf-48ae-ace8-ac8704e91784" (UID: "e0d8909f-05bf-48ae-ace8-ac8704e91784"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:29:39.090382 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.090351 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4snm\" (UniqueName: \"kubernetes.io/projected/e0d8909f-05bf-48ae-ace8-ac8704e91784-kube-api-access-w4snm\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:39.090382 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.090375 2560 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-ui-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:39.090557 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.090390 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-offline-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:39.090557 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.090404 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-registry-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:39.090557 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.090414 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e0d8909f-05bf-48ae-ace8-ac8704e91784-online-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:39.090557 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.090424 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e0d8909f-05bf-48ae-ace8-ac8704e91784-feast-data\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:29:39.101338 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.101304 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" path="/var/lib/kubelet/pods/bb59cd43-ff2b-42bb-a897-c591e24c1592/volumes" Jul 17 13:29:39.910829 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.910785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" event={"ID":"e0d8909f-05bf-48ae-ace8-ac8704e91784","Type":"ContainerDied","Data":"49f37a73d087e4800ffaf9cd96af08e343e81e26acdf0248bcf4c09c68367cf2"} Jul 17 13:29:39.911314 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.910850 2560 scope.go:117] "RemoveContainer" containerID="dc196c7d7a2bb246fc516d51461ea937221e880094a608e54e7b7454ed280ce3" Jul 17 13:29:39.911314 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.910799 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk" Jul 17 13:29:39.919350 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.919324 2560 scope.go:117] "RemoveContainer" containerID="a6bb4aff52593b5791bfba189e4b55b2aa6e92c35bee8b1c020aae224bbad519" Jul 17 13:29:39.926667 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.926650 2560 scope.go:117] "RemoveContainer" containerID="cf049b700b61b6df0e5f38dee41c78d4cf37030de8433d870837b1103863a474" Jul 17 13:29:39.931933 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.931708 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk"] Jul 17 13:29:39.936119 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.936098 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-849fccff88-x9gtk"] Jul 17 13:29:39.941358 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.941338 2560 scope.go:117] "RemoveContainer" containerID="3dfa2636ded0709afa9a2d297aa513294a8399bcb4a9ba864b4461444979f81a" Jul 17 13:29:39.948884 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.948865 2560 scope.go:117] "RemoveContainer" containerID="ff6dd013026ec7ea37abf0b50e6731bcf6303c86cbca30f5983cac03df38a97a" Jul 17 13:29:39.955638 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:39.955624 2560 scope.go:117] "RemoveContainer" containerID="3360843bf8c1d07643e930a9b55c4844033b41dbba19b39f128acf289f216320" Jul 17 13:29:41.100614 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:41.100540 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" path="/var/lib/kubelet/pods/e0d8909f-05bf-48ae-ace8-ac8704e91784/volumes" Jul 17 13:29:50.861648 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:50.861614 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.079808 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-864846d4f5-wrcw6"] Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080640 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="feast-apply" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080662 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="feast-apply" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080686 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080695 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080713 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080722 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080744 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="feast-apply" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080753 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="feast-apply" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080769 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="registry" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080778 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="registry" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080791 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="offline" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080800 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="offline" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080829 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="feast-init" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080837 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="feast-init" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080857 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080865 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080874 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="offline" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080882 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="offline" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080924 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="feast-init" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080932 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="feast-init" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080942 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.080951 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.081088 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="offline" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.081108 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="ui" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.081118 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="registry" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.081126 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="offline" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.081142 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="online" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.081150 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb59cd43-ff2b-42bb-a897-c591e24c1592" containerName="online" Jul 17 13:29:51.082646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.081160 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d8909f-05bf-48ae-ace8-ac8704e91784" containerName="ui" Jul 17 13:29:51.085932 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.085888 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.088680 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.088655 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wrcw6"] Jul 17 13:29:51.088872 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.088854 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\"" Jul 17 13:29:51.089180 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.088872 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\"" Jul 17 13:29:51.089180 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.088891 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-jxclv\"" Jul 17 13:29:51.090005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.089985 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\"" Jul 17 13:29:51.106758 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.106735 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/redis-69bbd798b6-rnkf8"] Jul 17 13:29:51.111371 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.111344 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:51.117031 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.116984 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-rnkf8" Jul 17 13:29:51.117499 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.117479 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-rnkf8"] Jul 17 13:29:51.162261 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.162225 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:51.186401 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.186372 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prlwb\" (UniqueName: \"kubernetes.io/projected/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-kube-api-access-prlwb\") pod \"postgres-864846d4f5-wrcw6\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.186515 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.186406 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-postgresdata\") pod \"postgres-864846d4f5-wrcw6\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.186515 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.186436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr48b\" (UniqueName: \"kubernetes.io/projected/d4aa291d-e75b-4a43-849c-50bf3bc19b5b-kube-api-access-gr48b\") pod \"redis-69bbd798b6-rnkf8\" (UID: \"d4aa291d-e75b-4a43-849c-50bf3bc19b5b\") " pod="test-ns-feast/redis-69bbd798b6-rnkf8" Jul 17 13:29:51.287173 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.287140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prlwb\" (UniqueName: \"kubernetes.io/projected/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-kube-api-access-prlwb\") pod \"postgres-864846d4f5-wrcw6\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.287319 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.287175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-postgresdata\") pod \"postgres-864846d4f5-wrcw6\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.287319 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.287199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr48b\" (UniqueName: \"kubernetes.io/projected/d4aa291d-e75b-4a43-849c-50bf3bc19b5b-kube-api-access-gr48b\") pod \"redis-69bbd798b6-rnkf8\" (UID: \"d4aa291d-e75b-4a43-849c-50bf3bc19b5b\") " pod="test-ns-feast/redis-69bbd798b6-rnkf8" Jul 17 13:29:51.287595 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.287576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-postgresdata\") pod \"postgres-864846d4f5-wrcw6\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.294860 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.294837 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prlwb\" (UniqueName: \"kubernetes.io/projected/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-kube-api-access-prlwb\") pod \"postgres-864846d4f5-wrcw6\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.294991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.294860 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr48b\" (UniqueName: \"kubernetes.io/projected/d4aa291d-e75b-4a43-849c-50bf3bc19b5b-kube-api-access-gr48b\") pod \"redis-69bbd798b6-rnkf8\" (UID: \"d4aa291d-e75b-4a43-849c-50bf3bc19b5b\") " pod="test-ns-feast/redis-69bbd798b6-rnkf8" Jul 17 13:29:51.411201 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.411109 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:29:51.426256 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.426233 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-rnkf8" Jul 17 13:29:51.544850 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.544822 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wrcw6"] Jul 17 13:29:51.546531 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:29:51.546501 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40e8cfde_e0b7_4e2a_9f47_f784119dac0c.slice/crio-51f88ecd8103eaad77f5fedd45d02d064ede8f5f2c9b8ef4fbb4457998574acf WatchSource:0}: Error finding container 51f88ecd8103eaad77f5fedd45d02d064ede8f5f2c9b8ef4fbb4457998574acf: Status 404 returned error can't find the container with id 51f88ecd8103eaad77f5fedd45d02d064ede8f5f2c9b8ef4fbb4457998574acf Jul 17 13:29:51.548429 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.548414 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 13:29:51.565497 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.565477 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/redis-69bbd798b6-rnkf8"] Jul 17 13:29:51.567317 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:29:51.567283 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4aa291d_e75b_4a43_849c_50bf3bc19b5b.slice/crio-c37b993100770470a7d6b1a5419e483cca177b13d7e610b71994e287cf2a8707 WatchSource:0}: Error finding container c37b993100770470a7d6b1a5419e483cca177b13d7e610b71994e287cf2a8707: Status 404 returned error can't find the container with id c37b993100770470a7d6b1a5419e483cca177b13d7e610b71994e287cf2a8707 Jul 17 13:29:51.962487 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.962454 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wrcw6" event={"ID":"40e8cfde-e0b7-4e2a-9f47-f784119dac0c","Type":"ContainerStarted","Data":"51f88ecd8103eaad77f5fedd45d02d064ede8f5f2c9b8ef4fbb4457998574acf"} Jul 17 13:29:51.963553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:51.963533 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-rnkf8" event={"ID":"d4aa291d-e75b-4a43-849c-50bf3bc19b5b","Type":"ContainerStarted","Data":"c37b993100770470a7d6b1a5419e483cca177b13d7e610b71994e287cf2a8707"} Jul 17 13:29:53.974852 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:53.974810 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-rnkf8" event={"ID":"d4aa291d-e75b-4a43-849c-50bf3bc19b5b","Type":"ContainerStarted","Data":"e62d354c15ed0e7b660a6406fcf6eddb2c0479027a0a2bf3084a8d472b383843"} Jul 17 13:29:53.989807 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:53.989756 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/redis-69bbd798b6-rnkf8" podStartSLOduration=1.344143474 podStartE2EDuration="2.989739063s" totalImagesPullingTime="1.645595589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:29:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:29:53.988815308 +0000 UTC m=+636.066928971" watchObservedRunningTime="2026-07-17 13:29:53.989739063 +0000 UTC m=+636.067852727" Jul 17 13:29:56.991787 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:56.991750 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wrcw6" event={"ID":"40e8cfde-e0b7-4e2a-9f47-f784119dac0c","Type":"ContainerStarted","Data":"13f10338998b598b1db52950d9c09f03e9aede45ef0f163669e106e93ccb7514"} Jul 17 13:29:57.006877 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:57.006813 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/postgres-864846d4f5-wrcw6" podStartSLOduration=1.004662847 podStartE2EDuration="6.006798986s" totalImagesPullingTime="5.002136139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:29:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:29:57.00597322 +0000 UTC m=+639.084086885" watchObservedRunningTime="2026-07-17 13:29:57.006798986 +0000 UTC m=+639.084912650" Jul 17 13:29:59.582691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.582660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:59.668948 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.668895 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:59.674158 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.674134 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd"] Jul 17 13:29:59.677988 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.677970 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.680392 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.680373 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\"" Jul 17 13:29:59.680492 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.680396 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\"" Jul 17 13:29:59.680492 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.680407 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-9kq56\"" Jul 17 13:29:59.680635 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.680617 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\"" Jul 17 13:29:59.680683 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.680663 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\"" Jul 17 13:29:59.687986 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.687965 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd"] Jul 17 13:29:59.764942 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.764895 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-registry-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.765097 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.764967 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.765097 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.765046 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-offline-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.765097 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.765069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggr6p\" (UniqueName: \"kubernetes.io/projected/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-kube-api-access-ggr6p\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.765250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.765159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-feast-data\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.805059 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.805033 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:59.866212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.866146 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-registry-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.866212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.866182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.866363 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.866255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-offline-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.866363 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:29:59.866279 2560 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found Jul 17 13:29:59.866363 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.866283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggr6p\" (UniqueName: \"kubernetes.io/projected/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-kube-api-access-ggr6p\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.866363 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:29:59.866324 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls podName:e7cbd9a6-8bea-469e-a44c-3bacd0531b59 nodeName:}" failed. No retries permitted until 2026-07-17 13:30:00.36630876 +0000 UTC m=+642.444422404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls") pod "feast-credit-scoring-85d8f6444d-rhqxd" (UID: "e7cbd9a6-8bea-469e-a44c-3bacd0531b59") : secret "feast-credit-scoring-online-tls" not found Jul 17 13:29:59.866551 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.866535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-feast-data\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.866854 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.866833 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-feast-data\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.868657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.868637 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-registry-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.868731 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.868700 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-offline-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.876578 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.876560 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggr6p\" (UniqueName: \"kubernetes.io/projected/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-kube-api-access-ggr6p\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:29:59.906636 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.906607 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:29:59.961931 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:29:59.961874 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:30:00.371611 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:00.371569 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:00.373978 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:00.373946 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls\") pod \"feast-credit-scoring-85d8f6444d-rhqxd\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:00.587854 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:00.587823 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:00.720586 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:00.720562 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd"] Jul 17 13:30:00.721264 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:30:00.721242 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cbd9a6_8bea_469e_a44c_3bacd0531b59.slice/crio-181512c5ce43af0472a33da63228a4ab5ed9994a4e247aae998c23de018fe66a WatchSource:0}: Error finding container 181512c5ce43af0472a33da63228a4ab5ed9994a4e247aae998c23de018fe66a: Status 404 returned error can't find the container with id 181512c5ce43af0472a33da63228a4ab5ed9994a4e247aae998c23de018fe66a Jul 17 13:30:01.010672 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:01.010636 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerStarted","Data":"16297e5704a60bbf8e59b59fe11ba5926d70214ae4f44866025d71eac449ca17"} Jul 17 13:30:01.010846 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:01.010690 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerStarted","Data":"181512c5ce43af0472a33da63228a4ab5ed9994a4e247aae998c23de018fe66a"} Jul 17 13:30:02.016227 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:02.016127 2560 generic.go:356] "Generic (PLEG): container finished" podID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerID="16297e5704a60bbf8e59b59fe11ba5926d70214ae4f44866025d71eac449ca17" exitCode=0 Jul 17 13:30:02.016585 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:02.016220 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerDied","Data":"16297e5704a60bbf8e59b59fe11ba5926d70214ae4f44866025d71eac449ca17"} Jul 17 13:30:03.022386 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:03.022349 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerStarted","Data":"d5d717baedbd6ccd6ba2c8a78956435b475ecd3415397c2abe0c52b3eee48e4b"} Jul 17 13:30:03.715207 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:03.715176 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:30:03.826157 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:03.826127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:30:09.048452 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:09.048416 2560 generic.go:356] "Generic (PLEG): container finished" podID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerID="d5d717baedbd6ccd6ba2c8a78956435b475ecd3415397c2abe0c52b3eee48e4b" exitCode=0 Jul 17 13:30:09.048868 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:09.048492 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerDied","Data":"d5d717baedbd6ccd6ba2c8a78956435b475ecd3415397c2abe0c52b3eee48e4b"} Jul 17 13:30:10.056484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:10.056444 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerStarted","Data":"07aa854c39d7aa1fc9e6c56b952b9b4552916c877135e2a528cf19370d0872a3"} Jul 17 13:30:10.056879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:10.056521 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerStarted","Data":"2d8c34b9d63c0da7e4cf1e2d38cec611956e792da4aa1532dbf3693d8bd58fd3"} Jul 17 13:30:10.056879 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:10.056549 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerStarted","Data":"58aaaef6125ecbbf3d1ec01e27792959ea17ce6a72a8a98aa02ccde733648f07"} Jul 17 13:30:10.082232 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:10.082184 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podStartSLOduration=11.082167398 podStartE2EDuration="11.082167398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:29:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:30:10.080432304 +0000 UTC m=+652.158546004" watchObservedRunningTime="2026-07-17 13:30:10.082167398 +0000 UTC m=+652.160281062" Jul 17 13:30:10.588920 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:10.588871 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:10.589092 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:10.588966 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:10.589092 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:10.588994 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:12.588442 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:12.588397 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:12.588970 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:12.588504 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:12.588970 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:12.588524 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:12.590200 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:12.590161 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused" Jul 17 13:30:12.590323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:12.590186 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Jul 17 13:30:12.590323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:12.590236 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Jul 17 13:30:15.588720 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:15.588668 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Jul 17 13:30:15.589760 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:15.589735 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:15.590712 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:15.590693 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:15.594777 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:15.594759 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:16.095693 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:16.095659 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:18.588537 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:18.588443 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused" Jul 17 13:30:21.589192 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:21.589158 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:21.590324 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:21.590289 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:30:21.828193 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:30:21.828157 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:31:02.146989 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:02.146945 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:31:02.197983 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:02.197949 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:31:20.176060 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:20.176025 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:31:25.161831 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.161799 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd"] Jul 17 13:31:25.162240 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.162032 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="registry" containerID="cri-o://58aaaef6125ecbbf3d1ec01e27792959ea17ce6a72a8a98aa02ccde733648f07" gracePeriod=30 Jul 17 13:31:25.162240 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.162082 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" containerID="cri-o://2d8c34b9d63c0da7e4cf1e2d38cec611956e792da4aa1532dbf3693d8bd58fd3" gracePeriod=30 Jul 17 13:31:25.162240 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.162065 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="offline" containerID="cri-o://07aa854c39d7aa1fc9e6c56b952b9b4552916c877135e2a528cf19370d0872a3" gracePeriod=30 Jul 17 13:31:25.177552 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.177528 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wrcw6"] Jul 17 13:31:25.177682 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.177666 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/postgres-864846d4f5-wrcw6" podUID="40e8cfde-e0b7-4e2a-9f47-f784119dac0c" containerName="postgres" containerID="cri-o://13f10338998b598b1db52950d9c09f03e9aede45ef0f163669e106e93ccb7514" gracePeriod=30 Jul 17 13:31:25.186188 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.186166 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-rnkf8"] Jul 17 13:31:25.186323 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.186308 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast/redis-69bbd798b6-rnkf8" podUID="d4aa291d-e75b-4a43-849c-50bf3bc19b5b" containerName="redis" containerID="cri-o://e62d354c15ed0e7b660a6406fcf6eddb2c0479027a0a2bf3084a8d472b383843" gracePeriod=30 Jul 17 13:31:25.253565 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.253541 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:31:25.386339 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.386314 2560 generic.go:356] "Generic (PLEG): container finished" podID="40e8cfde-e0b7-4e2a-9f47-f784119dac0c" containerID="13f10338998b598b1db52950d9c09f03e9aede45ef0f163669e106e93ccb7514" exitCode=0 Jul 17 13:31:25.386498 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.386385 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wrcw6" event={"ID":"40e8cfde-e0b7-4e2a-9f47-f784119dac0c","Type":"ContainerDied","Data":"13f10338998b598b1db52950d9c09f03e9aede45ef0f163669e106e93ccb7514"} Jul 17 13:31:25.388039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.388014 2560 generic.go:356] "Generic (PLEG): container finished" podID="d4aa291d-e75b-4a43-849c-50bf3bc19b5b" containerID="e62d354c15ed0e7b660a6406fcf6eddb2c0479027a0a2bf3084a8d472b383843" exitCode=0 Jul 17 13:31:25.388146 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.388079 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-rnkf8" event={"ID":"d4aa291d-e75b-4a43-849c-50bf3bc19b5b","Type":"ContainerDied","Data":"e62d354c15ed0e7b660a6406fcf6eddb2c0479027a0a2bf3084a8d472b383843"} Jul 17 13:31:25.439678 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.439660 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:31:25.442558 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.442539 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-rnkf8" Jul 17 13:31:25.526925 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.526874 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prlwb\" (UniqueName: \"kubernetes.io/projected/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-kube-api-access-prlwb\") pod \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " Jul 17 13:31:25.527096 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.526990 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-postgresdata\") pod \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\" (UID: \"40e8cfde-e0b7-4e2a-9f47-f784119dac0c\") " Jul 17 13:31:25.527096 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.527021 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gr48b\" (UniqueName: \"kubernetes.io/projected/d4aa291d-e75b-4a43-849c-50bf3bc19b5b-kube-api-access-gr48b\") pod \"d4aa291d-e75b-4a43-849c-50bf3bc19b5b\" (UID: \"d4aa291d-e75b-4a43-849c-50bf3bc19b5b\") " Jul 17 13:31:25.528993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.528959 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4aa291d-e75b-4a43-849c-50bf3bc19b5b-kube-api-access-gr48b" (OuterVolumeSpecName: "kube-api-access-gr48b") pod "d4aa291d-e75b-4a43-849c-50bf3bc19b5b" (UID: "d4aa291d-e75b-4a43-849c-50bf3bc19b5b"). InnerVolumeSpecName "kube-api-access-gr48b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:31:25.528993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.528973 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-kube-api-access-prlwb" (OuterVolumeSpecName: "kube-api-access-prlwb") pod "40e8cfde-e0b7-4e2a-9f47-f784119dac0c" (UID: "40e8cfde-e0b7-4e2a-9f47-f784119dac0c"). InnerVolumeSpecName "kube-api-access-prlwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:31:25.557273 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.557239 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "40e8cfde-e0b7-4e2a-9f47-f784119dac0c" (UID: "40e8cfde-e0b7-4e2a-9f47-f784119dac0c"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:31:25.627713 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.627675 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prlwb\" (UniqueName: \"kubernetes.io/projected/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-kube-api-access-prlwb\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:25.627713 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.627707 2560 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/40e8cfde-e0b7-4e2a-9f47-f784119dac0c-postgresdata\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:25.627713 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:25.627720 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gr48b\" (UniqueName: \"kubernetes.io/projected/d4aa291d-e75b-4a43-849c-50bf3bc19b5b-kube-api-access-gr48b\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:26.093615 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.093526 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Jul 17 13:31:26.393647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.393567 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/redis-69bbd798b6-rnkf8" Jul 17 13:31:26.393647 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.393603 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/redis-69bbd798b6-rnkf8" event={"ID":"d4aa291d-e75b-4a43-849c-50bf3bc19b5b","Type":"ContainerDied","Data":"c37b993100770470a7d6b1a5419e483cca177b13d7e610b71994e287cf2a8707"} Jul 17 13:31:26.394128 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.393652 2560 scope.go:117] "RemoveContainer" containerID="e62d354c15ed0e7b660a6406fcf6eddb2c0479027a0a2bf3084a8d472b383843" Jul 17 13:31:26.394980 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.394893 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-864846d4f5-wrcw6" Jul 17 13:31:26.394980 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.394939 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-864846d4f5-wrcw6" event={"ID":"40e8cfde-e0b7-4e2a-9f47-f784119dac0c","Type":"ContainerDied","Data":"51f88ecd8103eaad77f5fedd45d02d064ede8f5f2c9b8ef4fbb4457998574acf"} Jul 17 13:31:26.398085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.398063 2560 generic.go:356] "Generic (PLEG): container finished" podID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerID="2d8c34b9d63c0da7e4cf1e2d38cec611956e792da4aa1532dbf3693d8bd58fd3" exitCode=0 Jul 17 13:31:26.398197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.398110 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerDied","Data":"2d8c34b9d63c0da7e4cf1e2d38cec611956e792da4aa1532dbf3693d8bd58fd3"} Jul 17 13:31:26.402667 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.402552 2560 scope.go:117] "RemoveContainer" containerID="13f10338998b598b1db52950d9c09f03e9aede45ef0f163669e106e93ccb7514" Jul 17 13:31:26.416995 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.416967 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/redis-69bbd798b6-rnkf8"] Jul 17 13:31:26.419748 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.419731 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/redis-69bbd798b6-rnkf8"] Jul 17 13:31:26.429747 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.429728 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wrcw6"] Jul 17 13:31:26.430992 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:26.430975 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-864846d4f5-wrcw6"] Jul 17 13:31:27.100819 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:27.100789 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e8cfde-e0b7-4e2a-9f47-f784119dac0c" path="/var/lib/kubelet/pods/40e8cfde-e0b7-4e2a-9f47-f784119dac0c/volumes" Jul 17 13:31:27.101124 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:27.101111 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4aa291d-e75b-4a43-849c-50bf3bc19b5b" path="/var/lib/kubelet/pods/d4aa291d-e75b-4a43-849c-50bf3bc19b5b/volumes" Jul 17 13:31:36.093240 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:36.093175 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Jul 17 13:31:46.093390 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:46.093345 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused" Jul 17 13:31:46.093798 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:46.093498 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:31:55.528125 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.528089 2560 generic.go:356] "Generic (PLEG): container finished" podID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerID="07aa854c39d7aa1fc9e6c56b952b9b4552916c877135e2a528cf19370d0872a3" exitCode=137 Jul 17 13:31:55.528125 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.528114 2560 generic.go:356] "Generic (PLEG): container finished" podID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerID="58aaaef6125ecbbf3d1ec01e27792959ea17ce6a72a8a98aa02ccde733648f07" exitCode=137 Jul 17 13:31:55.528565 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.528148 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerDied","Data":"07aa854c39d7aa1fc9e6c56b952b9b4552916c877135e2a528cf19370d0872a3"} Jul 17 13:31:55.528565 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.528186 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerDied","Data":"58aaaef6125ecbbf3d1ec01e27792959ea17ce6a72a8a98aa02ccde733648f07"} Jul 17 13:31:55.590170 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.590133 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused" Jul 17 13:31:55.804069 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.804047 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:31:55.985467 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.985441 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-offline-tls\") pod \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " Jul 17 13:31:55.985467 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.985472 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-registry-tls\") pod \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " Jul 17 13:31:55.985697 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.985494 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls\") pod \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " Jul 17 13:31:55.985697 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.985523 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggr6p\" (UniqueName: \"kubernetes.io/projected/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-kube-api-access-ggr6p\") pod \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " Jul 17 13:31:55.985697 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.985580 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-feast-data\") pod \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\" (UID: \"e7cbd9a6-8bea-469e-a44c-3bacd0531b59\") " Jul 17 13:31:55.987584 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.987555 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e7cbd9a6-8bea-469e-a44c-3bacd0531b59" (UID: "e7cbd9a6-8bea-469e-a44c-3bacd0531b59"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:31:55.987721 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.987640 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-kube-api-access-ggr6p" (OuterVolumeSpecName: "kube-api-access-ggr6p") pod "e7cbd9a6-8bea-469e-a44c-3bacd0531b59" (UID: "e7cbd9a6-8bea-469e-a44c-3bacd0531b59"). InnerVolumeSpecName "kube-api-access-ggr6p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:31:55.988014 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.987997 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "e7cbd9a6-8bea-469e-a44c-3bacd0531b59" (UID: "e7cbd9a6-8bea-469e-a44c-3bacd0531b59"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:31:55.988079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.988017 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls" (OuterVolumeSpecName: "online-tls") pod "e7cbd9a6-8bea-469e-a44c-3bacd0531b59" (UID: "e7cbd9a6-8bea-469e-a44c-3bacd0531b59"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:31:55.993321 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:55.993296 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-feast-data" (OuterVolumeSpecName: "feast-data") pod "e7cbd9a6-8bea-469e-a44c-3bacd0531b59" (UID: "e7cbd9a6-8bea-469e-a44c-3bacd0531b59"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:31:56.086698 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.086602 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-feast-data\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:56.086698 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.086643 2560 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-offline-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:56.086698 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.086651 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-registry-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:56.086698 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.086659 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-online-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:56.086698 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.086667 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggr6p\" (UniqueName: \"kubernetes.io/projected/e7cbd9a6-8bea-469e-a44c-3bacd0531b59-kube-api-access-ggr6p\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:31:56.534682 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.534651 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" event={"ID":"e7cbd9a6-8bea-469e-a44c-3bacd0531b59","Type":"ContainerDied","Data":"181512c5ce43af0472a33da63228a4ab5ed9994a4e247aae998c23de018fe66a"} Jul 17 13:31:56.534682 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.534672 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd" Jul 17 13:31:56.535163 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.534714 2560 scope.go:117] "RemoveContainer" containerID="07aa854c39d7aa1fc9e6c56b952b9b4552916c877135e2a528cf19370d0872a3" Jul 17 13:31:56.543503 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.543488 2560 scope.go:117] "RemoveContainer" containerID="2d8c34b9d63c0da7e4cf1e2d38cec611956e792da4aa1532dbf3693d8bd58fd3" Jul 17 13:31:56.551257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.551244 2560 scope.go:117] "RemoveContainer" containerID="58aaaef6125ecbbf3d1ec01e27792959ea17ce6a72a8a98aa02ccde733648f07" Jul 17 13:31:56.556287 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.556265 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd"] Jul 17 13:31:56.559488 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.559465 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-85d8f6444d-rhqxd"] Jul 17 13:31:56.559702 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.559687 2560 scope.go:117] "RemoveContainer" containerID="d5d717baedbd6ccd6ba2c8a78956435b475ecd3415397c2abe0c52b3eee48e4b" Jul 17 13:31:56.566554 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:56.566540 2560 scope.go:117] "RemoveContainer" containerID="16297e5704a60bbf8e59b59fe11ba5926d70214ae4f44866025d71eac449ca17" Jul 17 13:31:57.099962 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:31:57.099905 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" path="/var/lib/kubelet/pods/e7cbd9a6-8bea-469e-a44c-3bacd0531b59/volumes" Jul 17 13:32:02.350396 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:32:02.350358 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:32:02.400522 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:32:02.400479 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:32:02.624336 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:32:02.624245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:32:02.728014 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:32:02.727982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:32:03.122962 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:32:03.122927 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:34:09.814176 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:34:09.814138 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:34:56.905356 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:34:56.905321 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:35:57.817602 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:57.817490 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:35:57.915806 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:57.915746 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:35:58.102313 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102235 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-cbjls"] Jul 17 13:35:58.102637 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102619 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="offline" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102639 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="offline" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102652 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="registry" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102662 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="registry" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102678 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="feast-apply" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102687 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="feast-apply" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102701 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102709 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102721 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="feast-init" Jul 17 13:35:58.102727 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102729 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="feast-init" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102761 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4aa291d-e75b-4a43-849c-50bf3bc19b5b" containerName="redis" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102770 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4aa291d-e75b-4a43-849c-50bf3bc19b5b" containerName="redis" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102779 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e8cfde-e0b7-4e2a-9f47-f784119dac0c" containerName="postgres" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102787 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e8cfde-e0b7-4e2a-9f47-f784119dac0c" containerName="postgres" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102860 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="online" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102874 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="offline" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102884 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e8cfde-e0b7-4e2a-9f47-f784119dac0c" containerName="postgres" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102894 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7cbd9a6-8bea-469e-a44c-3bacd0531b59" containerName="registry" Jul 17 13:35:58.103187 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.102924 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4aa291d-e75b-4a43-849c-50bf3bc19b5b" containerName="redis" Jul 17 13:35:58.105604 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.105582 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" Jul 17 13:35:58.108215 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.108183 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\"" Jul 17 13:35:58.108330 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.108214 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\"" Jul 17 13:35:58.108330 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.108248 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-9j2xx\"" Jul 17 13:35:58.114730 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.114706 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-cbjls"] Jul 17 13:35:58.129093 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.129066 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:35:58.130197 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.130176 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tbqp\" (UniqueName: \"kubernetes.io/projected/8ff7949e-bbfa-495a-89cc-40707225bb4c-kube-api-access-6tbqp\") pod \"redis-69bbd798b6-cbjls\" (UID: \"8ff7949e-bbfa-495a-89cc-40707225bb4c\") " pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" Jul 17 13:35:58.231360 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.231313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tbqp\" (UniqueName: \"kubernetes.io/projected/8ff7949e-bbfa-495a-89cc-40707225bb4c-kube-api-access-6tbqp\") pod \"redis-69bbd798b6-cbjls\" (UID: \"8ff7949e-bbfa-495a-89cc-40707225bb4c\") " pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" Jul 17 13:35:58.239259 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.239237 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tbqp\" (UniqueName: \"kubernetes.io/projected/8ff7949e-bbfa-495a-89cc-40707225bb4c-kube-api-access-6tbqp\") pod \"redis-69bbd798b6-cbjls\" (UID: \"8ff7949e-bbfa-495a-89cc-40707225bb4c\") " pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" Jul 17 13:35:58.425443 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.425419 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" Jul 17 13:35:58.547257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:58.547234 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-cbjls"] Jul 17 13:35:58.549698 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:35:58.549672 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ff7949e_bbfa_495a_89cc_40707225bb4c.slice/crio-4c1da064b151305fdcf72d72cc892934f18c5e2cfaa4b7190300054790c554fb WatchSource:0}: Error finding container 4c1da064b151305fdcf72d72cc892934f18c5e2cfaa4b7190300054790c554fb: Status 404 returned error can't find the container with id 4c1da064b151305fdcf72d72cc892934f18c5e2cfaa4b7190300054790c554fb Jul 17 13:35:59.481559 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:59.481524 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" event={"ID":"8ff7949e-bbfa-495a-89cc-40707225bb4c","Type":"ContainerStarted","Data":"8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974"} Jul 17 13:35:59.481559 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:59.481564 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" event={"ID":"8ff7949e-bbfa-495a-89cc-40707225bb4c","Type":"ContainerStarted","Data":"4c1da064b151305fdcf72d72cc892934f18c5e2cfaa4b7190300054790c554fb"} Jul 17 13:35:59.495688 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:35:59.495646 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" podStartSLOduration=1.495631923 podStartE2EDuration="1.495631923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:35:59.494700202 +0000 UTC m=+1001.572813868" watchObservedRunningTime="2026-07-17 13:35:59.495631923 +0000 UTC m=+1001.573745587" Jul 17 13:36:03.590336 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.590299 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-87gdx"] Jul 17 13:36:03.594302 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.594277 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:03.596791 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.596765 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\"" Jul 17 13:36:03.600660 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.600636 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-87gdx"] Jul 17 13:36:03.621637 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.621598 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:03.676715 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.676687 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djspm\" (UniqueName: \"kubernetes.io/projected/658970b9-40ec-49f0-9c7c-6f5ecdee4682-kube-api-access-djspm\") pod \"postgres-864846d4f5-87gdx\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:03.676715 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.676719 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/658970b9-40ec-49f0-9c7c-6f5ecdee4682-postgresdata\") pod \"postgres-864846d4f5-87gdx\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:03.777279 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.777240 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djspm\" (UniqueName: \"kubernetes.io/projected/658970b9-40ec-49f0-9c7c-6f5ecdee4682-kube-api-access-djspm\") pod \"postgres-864846d4f5-87gdx\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:03.777279 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.777281 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/658970b9-40ec-49f0-9c7c-6f5ecdee4682-postgresdata\") pod \"postgres-864846d4f5-87gdx\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:03.777609 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.777591 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/658970b9-40ec-49f0-9c7c-6f5ecdee4682-postgresdata\") pod \"postgres-864846d4f5-87gdx\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:03.784969 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.784943 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djspm\" (UniqueName: \"kubernetes.io/projected/658970b9-40ec-49f0-9c7c-6f5ecdee4682-kube-api-access-djspm\") pod \"postgres-864846d4f5-87gdx\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:03.904807 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:03.904724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:36:04.026230 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:04.026206 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-87gdx"] Jul 17 13:36:04.028690 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:36:04.028663 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod658970b9_40ec_49f0_9c7c_6f5ecdee4682.slice/crio-8d8d85f3c10e85a6b1b15425ae6a0a40d8aa6e9e36fc22cefe488aa63d2e50ad WatchSource:0}: Error finding container 8d8d85f3c10e85a6b1b15425ae6a0a40d8aa6e9e36fc22cefe488aa63d2e50ad: Status 404 returned error can't find the container with id 8d8d85f3c10e85a6b1b15425ae6a0a40d8aa6e9e36fc22cefe488aa63d2e50ad Jul 17 13:36:04.503472 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:04.503436 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" event={"ID":"658970b9-40ec-49f0-9c7c-6f5ecdee4682","Type":"ContainerStarted","Data":"f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35"} Jul 17 13:36:04.503635 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:04.503482 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" event={"ID":"658970b9-40ec-49f0-9c7c-6f5ecdee4682","Type":"ContainerStarted","Data":"8d8d85f3c10e85a6b1b15425ae6a0a40d8aa6e9e36fc22cefe488aa63d2e50ad"} Jul 17 13:36:04.518718 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:04.518670 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" podStartSLOduration=1.5186551179999999 podStartE2EDuration="1.518655118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:36:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:36:04.517412481 +0000 UTC m=+1006.595526148" watchObservedRunningTime="2026-07-17 13:36:04.518655118 +0000 UTC m=+1006.596768783" Jul 17 13:36:09.092229 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.092194 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:09.193506 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.193473 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:09.248504 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.248474 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq"] Jul 17 13:36:09.252152 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.252133 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.257165 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.257143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\"" Jul 17 13:36:09.257416 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.257394 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\"" Jul 17 13:36:09.257416 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.257409 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jul 17 13:36:09.258091 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.258063 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-pcffx\"" Jul 17 13:36:09.264368 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.264351 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq"] Jul 17 13:36:09.323065 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.323039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-online-tls\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.323065 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.323070 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.323244 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.323089 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-477vr\" (UniqueName: \"kubernetes.io/projected/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-kube-api-access-477vr\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.323244 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.323194 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-feast-data\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.334184 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.334159 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:09.424262 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.424230 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-online-tls\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.424262 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.424263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.424489 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.424278 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-477vr\" (UniqueName: \"kubernetes.io/projected/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-kube-api-access-477vr\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.424489 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.424319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-feast-data\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.424489 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:36:09.424380 2560 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found Jul 17 13:36:09.424489 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:36:09.424455 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls podName:a77aa4b1-81ac-4f8c-83ec-d5a0db72049a nodeName:}" failed. No retries permitted until 2026-07-17 13:36:09.924438213 +0000 UTC m=+1012.002551854 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls") pod "feast-credit-scoring-6db9985d85-fttsq" (UID: "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a") : secret "feast-credit-scoring-registry-tls" not found Jul 17 13:36:09.424700 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.424663 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-feast-data\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.426696 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.426677 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-online-tls\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.434046 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.434025 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-477vr\" (UniqueName: \"kubernetes.io/projected/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-kube-api-access-477vr\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.551865 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.551836 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:09.928076 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.928046 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:09.930407 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:09.930387 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls\") pod \"feast-credit-scoring-6db9985d85-fttsq\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:10.161603 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:10.161572 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:10.288039 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:10.288013 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq"] Jul 17 13:36:10.289866 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:36:10.289838 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77aa4b1_81ac_4f8c_83ec_d5a0db72049a.slice/crio-d0e99ca594c8679d983369820bf4d099eb7cef67b6e5aed739885acba43b2b87 WatchSource:0}: Error finding container d0e99ca594c8679d983369820bf4d099eb7cef67b6e5aed739885acba43b2b87: Status 404 returned error can't find the container with id d0e99ca594c8679d983369820bf4d099eb7cef67b6e5aed739885acba43b2b87 Jul 17 13:36:10.531144 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:10.531055 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerStarted","Data":"94d368265de71ef122242882de4ddd839593ce272478c192f0e3611c900978a6"} Jul 17 13:36:10.531144 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:10.531117 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerStarted","Data":"d0e99ca594c8679d983369820bf4d099eb7cef67b6e5aed739885acba43b2b87"} Jul 17 13:36:11.536589 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:11.536559 2560 generic.go:356] "Generic (PLEG): container finished" podID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerID="94d368265de71ef122242882de4ddd839593ce272478c192f0e3611c900978a6" exitCode=0 Jul 17 13:36:11.537005 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:11.536645 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerDied","Data":"94d368265de71ef122242882de4ddd839593ce272478c192f0e3611c900978a6"} Jul 17 13:36:12.543337 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:12.543296 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerStarted","Data":"fa65cccfc1af99c972a4bbfe9190229611a80b079f1083c230271806e42569d9"} Jul 17 13:36:18.570081 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:18.570050 2560 generic.go:356] "Generic (PLEG): container finished" podID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerID="fa65cccfc1af99c972a4bbfe9190229611a80b079f1083c230271806e42569d9" exitCode=0 Jul 17 13:36:18.570507 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:18.570090 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerDied","Data":"fa65cccfc1af99c972a4bbfe9190229611a80b079f1083c230271806e42569d9"} Jul 17 13:36:18.574779 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:18.574720 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\"" Jul 17 13:36:19.577858 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:19.577819 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerStarted","Data":"00a139ee2c2189f48169ba64ae5dc6aab829420f1598dde8d0812e894e1b50ef"} Jul 17 13:36:19.578361 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:19.577872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerStarted","Data":"abd885db9c9e6be3ef8930ba7dfb0f773f62fd1e959ae27a7e6aee95f6a55c44"} Jul 17 13:36:19.599493 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:19.599433 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podStartSLOduration=10.599419522 podStartE2EDuration="10.599419522s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:36:19.596948215 +0000 UTC m=+1021.675061879" watchObservedRunningTime="2026-07-17 13:36:19.599419522 +0000 UTC m=+1021.677533186" Jul 17 13:36:20.161925 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:20.161881 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:20.162082 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:20.161968 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:22.162761 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:22.162718 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:22.163271 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:22.162786 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:22.164169 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:22.164140 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Jul 17 13:36:22.164314 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:22.164174 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Jul 17 13:36:25.162696 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:25.162641 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Jul 17 13:36:25.169182 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:25.169156 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:25.172527 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:25.172503 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:28.162117 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:28.162077 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused" Jul 17 13:36:31.163217 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:31.163182 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:31.163894 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:31.163873 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:36:31.410049 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:31.410010 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:35.040850 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.040807 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:35.142079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.142045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:35.193058 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.193019 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x"] Jul 17 13:36:35.197821 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.197801 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.200446 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.200426 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\"" Jul 17 13:36:35.200565 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.200504 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\"" Jul 17 13:36:35.200630 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.200607 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\"" Jul 17 13:36:35.200681 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.200659 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-ks2zw\"" Jul 17 13:36:35.207387 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.207368 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x"] Jul 17 13:36:35.212177 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.212152 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:35.359222 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.359122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-online-tls\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.359222 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.359191 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs6jk\" (UniqueName: \"kubernetes.io/projected/2ed5698f-f362-4385-8774-7090ab24b536-kube-api-access-qs6jk\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.359457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.359275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-registry-tls\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.359457 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.359315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed5698f-f362-4385-8774-7090ab24b536-feast-data\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.413920 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.413888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:36:35.460190 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.460141 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-registry-tls\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.460190 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.460203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed5698f-f362-4385-8774-7090ab24b536-feast-data\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.460434 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.460242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-online-tls\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.460434 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.460284 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs6jk\" (UniqueName: \"kubernetes.io/projected/2ed5698f-f362-4385-8774-7090ab24b536-kube-api-access-qs6jk\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.460672 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.460649 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed5698f-f362-4385-8774-7090ab24b536-feast-data\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.462655 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.462632 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-online-tls\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.462802 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.462784 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-registry-tls\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.468901 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.468872 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs6jk\" (UniqueName: \"kubernetes.io/projected/2ed5698f-f362-4385-8774-7090ab24b536-kube-api-access-qs6jk\") pod \"feast-driver-ranking-69bf4fdc9d-qzm6x\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.508079 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.508035 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:35.647111 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:35.647086 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x"] Jul 17 13:36:35.649127 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:36:35.649098 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed5698f_f362_4385_8774_7090ab24b536.slice/crio-4184c98b9eef0b56406786a11d0e95790cd52f1ab16f61f1e56c1a8d8ca4ae6e WatchSource:0}: Error finding container 4184c98b9eef0b56406786a11d0e95790cd52f1ab16f61f1e56c1a8d8ca4ae6e: Status 404 returned error can't find the container with id 4184c98b9eef0b56406786a11d0e95790cd52f1ab16f61f1e56c1a8d8ca4ae6e Jul 17 13:36:36.658346 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:36.658303 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerStarted","Data":"4b0a9da1549434f4cc5679da4ec0f045a44289bd815086c78a2defa9b5f8d169"} Jul 17 13:36:36.658787 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:36.658402 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerStarted","Data":"4184c98b9eef0b56406786a11d0e95790cd52f1ab16f61f1e56c1a8d8ca4ae6e"} Jul 17 13:36:40.678978 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:40.678947 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ed5698f-f362-4385-8774-7090ab24b536" containerID="4b0a9da1549434f4cc5679da4ec0f045a44289bd815086c78a2defa9b5f8d169" exitCode=0 Jul 17 13:36:40.679344 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:40.678984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerDied","Data":"4b0a9da1549434f4cc5679da4ec0f045a44289bd815086c78a2defa9b5f8d169"} Jul 17 13:36:41.684724 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:41.684691 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerStarted","Data":"1c203e5078af07ec2da641f0583c5d1d7a238cb5420524466119b789199b917a"} Jul 17 13:36:47.713433 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:47.713398 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ed5698f-f362-4385-8774-7090ab24b536" containerID="1c203e5078af07ec2da641f0583c5d1d7a238cb5420524466119b789199b917a" exitCode=0 Jul 17 13:36:47.713964 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:47.713451 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerDied","Data":"1c203e5078af07ec2da641f0583c5d1d7a238cb5420524466119b789199b917a"} Jul 17 13:36:48.720116 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:48.720077 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerStarted","Data":"848c59263c09aec4e33b94a72c198fcf0ef8a38b2eb969f21f3d8d2e3156041d"} Jul 17 13:36:48.720591 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:48.720143 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerStarted","Data":"672aba001ee3b755a1109cf2100a54cabe4b92d6709b19d85be29d3416bdc562"} Jul 17 13:36:48.743011 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:48.742948 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podStartSLOduration=13.742925731 podStartE2EDuration="13.742925731s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:36:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:36:48.739887047 +0000 UTC m=+1050.818000736" watchObservedRunningTime="2026-07-17 13:36:48.742925731 +0000 UTC m=+1050.821039394" Jul 17 13:36:50.508927 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:50.508879 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:50.509316 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:50.508961 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:50.510624 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:50.510591 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Jul 17 13:36:50.510740 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:50.510605 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Jul 17 13:36:53.508721 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:53.508668 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Jul 17 13:36:53.974116 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:53.974079 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:53.974626 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:53.974601 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:53.977628 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:53.977602 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:55.509114 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:55.509082 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:56.508573 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:56.508523 2560 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused" Jul 17 13:36:59.509716 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:59.509684 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:36:59.510463 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:36:59.510446 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:38:11.581502 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:11.581458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:13.559213 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:13.559177 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:13.855526 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:13.855432 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:14.043944 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.043887 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:14.073245 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.073201 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq"] Jul 17 13:38:14.073522 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.073494 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="registry" containerID="cri-o://abd885db9c9e6be3ef8930ba7dfb0f773f62fd1e959ae27a7e6aee95f6a55c44" gracePeriod=30 Jul 17 13:38:14.073611 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.073558 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" containerID="cri-o://00a139ee2c2189f48169ba64ae5dc6aab829420f1598dde8d0812e894e1b50ef" gracePeriod=30 Jul 17 13:38:14.077597 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.077569 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x"] Jul 17 13:38:14.077973 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.077820 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="registry" containerID="cri-o://672aba001ee3b755a1109cf2100a54cabe4b92d6709b19d85be29d3416bdc562" gracePeriod=30 Jul 17 13:38:14.077973 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.077883 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" containerID="cri-o://848c59263c09aec4e33b94a72c198fcf0ef8a38b2eb969f21f3d8d2e3156041d" gracePeriod=30 Jul 17 13:38:14.104252 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:14.104220 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:15.110067 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:15.109987 2560 generic.go:356] "Generic (PLEG): container finished" podID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerID="00a139ee2c2189f48169ba64ae5dc6aab829420f1598dde8d0812e894e1b50ef" exitCode=0 Jul 17 13:38:15.110067 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:15.110057 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerDied","Data":"00a139ee2c2189f48169ba64ae5dc6aab829420f1598dde8d0812e894e1b50ef"} Jul 17 13:38:15.112338 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:15.112318 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ed5698f-f362-4385-8774-7090ab24b536" containerID="848c59263c09aec4e33b94a72c198fcf0ef8a38b2eb969f21f3d8d2e3156041d" exitCode=0 Jul 17 13:38:15.112473 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:15.112360 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerDied","Data":"848c59263c09aec4e33b94a72c198fcf0ef8a38b2eb969f21f3d8d2e3156041d"} Jul 17 13:38:15.170428 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:15.170387 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Jul 17 13:38:16.541864 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.541828 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-87gdx"] Jul 17 13:38:16.542279 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.542014 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" podUID="658970b9-40ec-49f0-9c7c-6f5ecdee4682" containerName="postgres" containerID="cri-o://f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35" gracePeriod=30 Jul 17 13:38:16.547071 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.547043 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-cbjls"] Jul 17 13:38:16.547252 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.547231 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" podUID="8ff7949e-bbfa-495a-89cc-40707225bb4c" containerName="redis" containerID="cri-o://8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974" gracePeriod=30 Jul 17 13:38:16.599200 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.599153 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:16.811208 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.811188 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:38:16.814080 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.814061 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" Jul 17 13:38:16.862700 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.862673 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6tbqp\" (UniqueName: \"kubernetes.io/projected/8ff7949e-bbfa-495a-89cc-40707225bb4c-kube-api-access-6tbqp\") pod \"8ff7949e-bbfa-495a-89cc-40707225bb4c\" (UID: \"8ff7949e-bbfa-495a-89cc-40707225bb4c\") " Jul 17 13:38:16.862884 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.862724 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/658970b9-40ec-49f0-9c7c-6f5ecdee4682-postgresdata\") pod \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " Jul 17 13:38:16.862884 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.862765 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djspm\" (UniqueName: \"kubernetes.io/projected/658970b9-40ec-49f0-9c7c-6f5ecdee4682-kube-api-access-djspm\") pod \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\" (UID: \"658970b9-40ec-49f0-9c7c-6f5ecdee4682\") " Jul 17 13:38:16.864984 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.864949 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff7949e-bbfa-495a-89cc-40707225bb4c-kube-api-access-6tbqp" (OuterVolumeSpecName: "kube-api-access-6tbqp") pod "8ff7949e-bbfa-495a-89cc-40707225bb4c" (UID: "8ff7949e-bbfa-495a-89cc-40707225bb4c"). InnerVolumeSpecName "kube-api-access-6tbqp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:38:16.865103 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.865056 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658970b9-40ec-49f0-9c7c-6f5ecdee4682-kube-api-access-djspm" (OuterVolumeSpecName: "kube-api-access-djspm") pod "658970b9-40ec-49f0-9c7c-6f5ecdee4682" (UID: "658970b9-40ec-49f0-9c7c-6f5ecdee4682"). InnerVolumeSpecName "kube-api-access-djspm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:38:16.892013 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.891978 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658970b9-40ec-49f0-9c7c-6f5ecdee4682-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "658970b9-40ec-49f0-9c7c-6f5ecdee4682" (UID: "658970b9-40ec-49f0-9c7c-6f5ecdee4682"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:38:16.963691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.963656 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tbqp\" (UniqueName: \"kubernetes.io/projected/8ff7949e-bbfa-495a-89cc-40707225bb4c-kube-api-access-6tbqp\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:16.963691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.963687 2560 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/658970b9-40ec-49f0-9c7c-6f5ecdee4682-postgresdata\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:16.963691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:16.963697 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djspm\" (UniqueName: \"kubernetes.io/projected/658970b9-40ec-49f0-9c7c-6f5ecdee4682-kube-api-access-djspm\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:17.127287 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.127248 2560 generic.go:356] "Generic (PLEG): container finished" podID="658970b9-40ec-49f0-9c7c-6f5ecdee4682" containerID="f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35" exitCode=0 Jul 17 13:38:17.127462 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.127330 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" event={"ID":"658970b9-40ec-49f0-9c7c-6f5ecdee4682","Type":"ContainerDied","Data":"f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35"} Jul 17 13:38:17.127462 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.127369 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" Jul 17 13:38:17.127462 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.127378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-864846d4f5-87gdx" event={"ID":"658970b9-40ec-49f0-9c7c-6f5ecdee4682","Type":"ContainerDied","Data":"8d8d85f3c10e85a6b1b15425ae6a0a40d8aa6e9e36fc22cefe488aa63d2e50ad"} Jul 17 13:38:17.127462 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.127401 2560 scope.go:117] "RemoveContainer" containerID="f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35" Jul 17 13:38:17.128870 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.128850 2560 generic.go:356] "Generic (PLEG): container finished" podID="8ff7949e-bbfa-495a-89cc-40707225bb4c" containerID="8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974" exitCode=0 Jul 17 13:38:17.128959 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.128875 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" event={"ID":"8ff7949e-bbfa-495a-89cc-40707225bb4c","Type":"ContainerDied","Data":"8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974"} Jul 17 13:38:17.128959 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.128927 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" event={"ID":"8ff7949e-bbfa-495a-89cc-40707225bb4c","Type":"ContainerDied","Data":"4c1da064b151305fdcf72d72cc892934f18c5e2cfaa4b7190300054790c554fb"} Jul 17 13:38:17.128959 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.128934 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-69bbd798b6-cbjls" Jul 17 13:38:17.136034 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.136007 2560 scope.go:117] "RemoveContainer" containerID="f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35" Jul 17 13:38:17.136361 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:38:17.136332 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35\": container with ID starting with f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35 not found: ID does not exist" containerID="f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35" Jul 17 13:38:17.136417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.136382 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35"} err="failed to get container status \"f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35\": rpc error: code = NotFound desc = could not find container \"f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35\": container with ID starting with f5d7b2e6f02f427068dd682a0f37fcdae97da43fb399bbe1a0f089d374234f35 not found: ID does not exist" Jul 17 13:38:17.136417 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.136409 2560 scope.go:117] "RemoveContainer" containerID="8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974" Jul 17 13:38:17.144080 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.144059 2560 scope.go:117] "RemoveContainer" containerID="8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974" Jul 17 13:38:17.144235 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.144215 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-87gdx"] Jul 17 13:38:17.144388 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:38:17.144351 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974\": container with ID starting with 8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974 not found: ID does not exist" containerID="8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974" Jul 17 13:38:17.144448 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.144390 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974"} err="failed to get container status \"8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974\": rpc error: code = NotFound desc = could not find container \"8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974\": container with ID starting with 8427612e96a06c06932d11f1847b47292ae1c2d0862b4770171ae3a328655974 not found: ID does not exist" Jul 17 13:38:17.147314 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.147296 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-864846d4f5-87gdx"] Jul 17 13:38:17.157229 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.157200 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-cbjls"] Jul 17 13:38:17.160590 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:17.160569 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-69bbd798b6-cbjls"] Jul 17 13:38:18.919798 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:18.919725 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:18.970118 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:18.970088 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:19.102250 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:19.102224 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="658970b9-40ec-49f0-9c7c-6f5ecdee4682" path="/var/lib/kubelet/pods/658970b9-40ec-49f0-9c7c-6f5ecdee4682/volumes" Jul 17 13:38:19.102539 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:19.102527 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff7949e-bbfa-495a-89cc-40707225bb4c" path="/var/lib/kubelet/pods/8ff7949e-bbfa-495a-89cc-40707225bb4c/volumes" Jul 17 13:38:23.975191 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:23.975146 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Jul 17 13:38:25.170314 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:25.170281 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Jul 17 13:38:26.375316 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:26.375279 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:26.477063 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:26.477027 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:26.847714 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:26.847684 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:27.461502 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:27.461469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:33.975034 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:33.974988 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Jul 17 13:38:35.170273 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:35.170233 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused" Jul 17 13:38:35.170722 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:35.170343 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:38:43.055904 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.055828 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:43.156304 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.156273 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:43.432019 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.431988 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:43.519511 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.519478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:43.575203 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.575173 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:43.649390 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.649359 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:43.747929 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.747840 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:43.975004 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.974965 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused" Jul 17 13:38:43.975181 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:43.975098 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:38:44.240884 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.240856 2560 generic.go:356] "Generic (PLEG): container finished" podID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerID="abd885db9c9e6be3ef8930ba7dfb0f773f62fd1e959ae27a7e6aee95f6a55c44" exitCode=137 Jul 17 13:38:44.241349 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.240935 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerDied","Data":"abd885db9c9e6be3ef8930ba7dfb0f773f62fd1e959ae27a7e6aee95f6a55c44"} Jul 17 13:38:44.243099 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.243079 2560 generic.go:356] "Generic (PLEG): container finished" podID="2ed5698f-f362-4385-8774-7090ab24b536" containerID="672aba001ee3b755a1109cf2100a54cabe4b92d6709b19d85be29d3416bdc562" exitCode=137 Jul 17 13:38:44.243210 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.243136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerDied","Data":"672aba001ee3b755a1109cf2100a54cabe4b92d6709b19d85be29d3416bdc562"} Jul 17 13:38:44.747702 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.747671 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:38:44.750355 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.750337 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:38:44.794232 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.794201 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed5698f-f362-4385-8774-7090ab24b536-feast-data\") pod \"2ed5698f-f362-4385-8774-7090ab24b536\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " Jul 17 13:38:44.794436 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.794266 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-online-tls\") pod \"2ed5698f-f362-4385-8774-7090ab24b536\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " Jul 17 13:38:44.794436 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.794329 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-registry-tls\") pod \"2ed5698f-f362-4385-8774-7090ab24b536\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " Jul 17 13:38:44.794436 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.794368 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs6jk\" (UniqueName: \"kubernetes.io/projected/2ed5698f-f362-4385-8774-7090ab24b536-kube-api-access-qs6jk\") pod \"2ed5698f-f362-4385-8774-7090ab24b536\" (UID: \"2ed5698f-f362-4385-8774-7090ab24b536\") " Jul 17 13:38:44.794825 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.794803 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ed5698f-f362-4385-8774-7090ab24b536-feast-data" (OuterVolumeSpecName: "feast-data") pod "2ed5698f-f362-4385-8774-7090ab24b536" (UID: "2ed5698f-f362-4385-8774-7090ab24b536"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:38:44.796354 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.796323 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-online-tls" (OuterVolumeSpecName: "online-tls") pod "2ed5698f-f362-4385-8774-7090ab24b536" (UID: "2ed5698f-f362-4385-8774-7090ab24b536"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:38:44.796454 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.796429 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ed5698f-f362-4385-8774-7090ab24b536-kube-api-access-qs6jk" (OuterVolumeSpecName: "kube-api-access-qs6jk") pod "2ed5698f-f362-4385-8774-7090ab24b536" (UID: "2ed5698f-f362-4385-8774-7090ab24b536"). InnerVolumeSpecName "kube-api-access-qs6jk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:38:44.796721 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.796703 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2ed5698f-f362-4385-8774-7090ab24b536" (UID: "2ed5698f-f362-4385-8774-7090ab24b536"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:38:44.895731 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895644 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-477vr\" (UniqueName: \"kubernetes.io/projected/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-kube-api-access-477vr\") pod \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " Jul 17 13:38:44.895731 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895706 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-online-tls\") pod \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " Jul 17 13:38:44.895993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895739 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls\") pod \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " Jul 17 13:38:44.895993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895757 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-feast-data\") pod \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\" (UID: \"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a\") " Jul 17 13:38:44.895993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895886 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs6jk\" (UniqueName: \"kubernetes.io/projected/2ed5698f-f362-4385-8774-7090ab24b536-kube-api-access-qs6jk\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:44.895993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895901 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2ed5698f-f362-4385-8774-7090ab24b536-feast-data\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:44.895993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895928 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-online-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:44.895993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.895936 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2ed5698f-f362-4385-8774-7090ab24b536-registry-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:44.897865 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.897833 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" (UID: "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:38:44.897987 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.897876 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-online-tls" (OuterVolumeSpecName: "online-tls") pod "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" (UID: "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 13:38:44.897987 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.897883 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-kube-api-access-477vr" (OuterVolumeSpecName: "kube-api-access-477vr") pod "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" (UID: "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a"). InnerVolumeSpecName "kube-api-access-477vr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:38:44.906272 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.906244 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-feast-data" (OuterVolumeSpecName: "feast-data") pod "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" (UID: "a77aa4b1-81ac-4f8c-83ec-d5a0db72049a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:38:44.996584 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.996550 2560 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-online-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:44.996584 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.996578 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-registry-tls\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:44.996584 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.996586 2560 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-feast-data\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:44.996584 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:44.996594 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-477vr\" (UniqueName: \"kubernetes.io/projected/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a-kube-api-access-477vr\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:38:45.249220 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.249180 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" event={"ID":"a77aa4b1-81ac-4f8c-83ec-d5a0db72049a","Type":"ContainerDied","Data":"d0e99ca594c8679d983369820bf4d099eb7cef67b6e5aed739885acba43b2b87"} Jul 17 13:38:45.249634 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.249251 2560 scope.go:117] "RemoveContainer" containerID="00a139ee2c2189f48169ba64ae5dc6aab829420f1598dde8d0812e894e1b50ef" Jul 17 13:38:45.249634 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.249190 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq" Jul 17 13:38:45.251856 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.251827 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" event={"ID":"2ed5698f-f362-4385-8774-7090ab24b536","Type":"ContainerDied","Data":"4184c98b9eef0b56406786a11d0e95790cd52f1ab16f61f1e56c1a8d8ca4ae6e"} Jul 17 13:38:45.252048 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.251877 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x" Jul 17 13:38:45.258245 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.258231 2560 scope.go:117] "RemoveContainer" containerID="abd885db9c9e6be3ef8930ba7dfb0f773f62fd1e959ae27a7e6aee95f6a55c44" Jul 17 13:38:45.265726 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.265704 2560 scope.go:117] "RemoveContainer" containerID="fa65cccfc1af99c972a4bbfe9190229611a80b079f1083c230271806e42569d9" Jul 17 13:38:45.272394 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.272368 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq"] Jul 17 13:38:45.273002 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.272985 2560 scope.go:117] "RemoveContainer" containerID="94d368265de71ef122242882de4ddd839593ce272478c192f0e3611c900978a6" Jul 17 13:38:45.276646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.276625 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6db9985d85-fttsq"] Jul 17 13:38:45.280698 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.280682 2560 scope.go:117] "RemoveContainer" containerID="848c59263c09aec4e33b94a72c198fcf0ef8a38b2eb969f21f3d8d2e3156041d" Jul 17 13:38:45.287673 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.287654 2560 scope.go:117] "RemoveContainer" containerID="672aba001ee3b755a1109cf2100a54cabe4b92d6709b19d85be29d3416bdc562" Jul 17 13:38:45.287832 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.287813 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x"] Jul 17 13:38:45.291874 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.291858 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-69bf4fdc9d-qzm6x"] Jul 17 13:38:45.294864 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.294844 2560 scope.go:117] "RemoveContainer" containerID="1c203e5078af07ec2da641f0583c5d1d7a238cb5420524466119b789199b917a" Jul 17 13:38:45.301543 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:45.301528 2560 scope.go:117] "RemoveContainer" containerID="4b0a9da1549434f4cc5679da4ec0f045a44289bd815086c78a2defa9b5f8d169" Jul 17 13:38:47.101430 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:47.101398 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ed5698f-f362-4385-8774-7090ab24b536" path="/var/lib/kubelet/pods/2ed5698f-f362-4385-8774-7090ab24b536/volumes" Jul 17 13:38:47.101942 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:47.101927 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" path="/var/lib/kubelet/pods/a77aa4b1-81ac-4f8c-83ec-d5a0db72049a/volumes" Jul 17 13:38:54.129401 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:54.129361 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:54.229128 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:54.229098 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:38:54.329735 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:38:54.329702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:39:24.929804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:39:24.929766 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:22.916045 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:22.916009 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:23.017584 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:23.017550 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:23.118449 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:23.118413 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:23.186822 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:23.186791 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:23.266958 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:23.266927 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:23.385884 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:23.385850 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:23.486656 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:23.486582 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:23.587410 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:23.587379 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:27.003702 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:27.003670 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:40:35.883308 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:40:35.883268 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:11.085648 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:11.085575 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:48.633518 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:48.633432 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:48.734037 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:48.733998 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:48.898715 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:48.898639 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:48.966387 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:48.966352 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:49.232461 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:49.232427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:49.412936 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:49.412889 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:41:49.514024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:41:49.513897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:10.417035 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:10.416994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:10.485810 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:10.485783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:10.570048 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:10.569977 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:10.654471 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:10.654441 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:10.753009 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:10.752960 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:15.741592 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:15.741562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:54.082257 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:54.082216 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:54.132691 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:54.132660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:58.788341 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:58.788302 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:43:58.839053 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:43:58.839026 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:17.587582 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:17.587547 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:17.687139 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:17.687109 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:20.898379 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:20.898346 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:20.978814 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:20.978771 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:21.147764 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:21.147734 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:21.357094 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:21.357066 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:21.458024 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:21.457993 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:44:21.561255 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:44:21.561222 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:41.301448 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:45:41.301403 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:45:43.873622 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:45:43.873577 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:06.694008 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:47:06.693973 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:06.792145 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:47:06.792122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:06.921537 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:47:06.921496 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:07.020693 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:47:07.020609 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:07.195810 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:47:07.195773 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:07.348762 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:47:07.348672 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:47:07.449626 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:47:07.449581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:18.555507 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:18.555469 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:27.359407 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:27.359368 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:32.051828 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:32.051796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:32.105867 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:32.105830 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:32.196485 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:32.196447 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:32.296122 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:32.296090 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:32.344651 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:32.344531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:48:37.388700 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:48:37.388648 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:14.869407 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:14.869366 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:14.999740 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:14.999708 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:20.397945 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.397894 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:20.446850 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.446825 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:20.691378 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691347 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gbbsf/must-gather-5cn4l"] Jul 17 13:49:20.691690 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691678 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="feast-init" Jul 17 13:49:20.691736 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691692 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="feast-init" Jul 17 13:49:20.691736 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691705 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="feast-apply" Jul 17 13:49:20.691736 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691710 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="feast-apply" Jul 17 13:49:20.691736 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691723 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="658970b9-40ec-49f0-9c7c-6f5ecdee4682" containerName="postgres" Jul 17 13:49:20.691736 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691729 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="658970b9-40ec-49f0-9c7c-6f5ecdee4682" containerName="postgres" Jul 17 13:49:20.691736 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691736 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="feast-init" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691742 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="feast-init" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691748 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="registry" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691753 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="registry" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691762 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="feast-apply" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691768 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="feast-apply" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691773 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691779 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691785 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff7949e-bbfa-495a-89cc-40707225bb4c" containerName="redis" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691790 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff7949e-bbfa-495a-89cc-40707225bb4c" containerName="redis" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691801 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="registry" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691805 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="registry" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691812 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691817 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691859 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff7949e-bbfa-495a-89cc-40707225bb4c" containerName="redis" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691864 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="online" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691870 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="online" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691875 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a77aa4b1-81ac-4f8c-83ec-d5a0db72049a" containerName="registry" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691881 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ed5698f-f362-4385-8774-7090ab24b536" containerName="registry" Jul 17 13:49:20.691943 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.691888 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="658970b9-40ec-49f0-9c7c-6f5ecdee4682" containerName="postgres" Jul 17 13:49:20.694212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.694184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:20.694978 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.694963 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:20.697518 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.697495 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gbbsf\"/\"openshift-service-ca.crt\"" Jul 17 13:49:20.697518 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.697510 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gbbsf\"/\"default-dockercfg-7x4h4\"" Jul 17 13:49:20.698875 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.698855 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gbbsf\"/\"kube-root-ca.crt\"" Jul 17 13:49:20.702262 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.702241 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gbbsf/must-gather-5cn4l"] Jul 17 13:49:20.742965 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.742937 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:20.802398 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.802350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa895b35-c37e-4069-ba0a-306abd972b90-must-gather-output\") pod \"must-gather-5cn4l\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:20.802398 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.802398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl9g5\" (UniqueName: \"kubernetes.io/projected/aa895b35-c37e-4069-ba0a-306abd972b90-kube-api-access-pl9g5\") pod \"must-gather-5cn4l\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:20.903212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.903183 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa895b35-c37e-4069-ba0a-306abd972b90-must-gather-output\") pod \"must-gather-5cn4l\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:20.903212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.903214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl9g5\" (UniqueName: \"kubernetes.io/projected/aa895b35-c37e-4069-ba0a-306abd972b90-kube-api-access-pl9g5\") pod \"must-gather-5cn4l\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:20.903519 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.903502 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa895b35-c37e-4069-ba0a-306abd972b90-must-gather-output\") pod \"must-gather-5cn4l\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:20.910762 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:20.910736 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl9g5\" (UniqueName: \"kubernetes.io/projected/aa895b35-c37e-4069-ba0a-306abd972b90-kube-api-access-pl9g5\") pod \"must-gather-5cn4l\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:21.016128 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:21.016045 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:21.140566 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:21.140542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gbbsf/must-gather-5cn4l"] Jul 17 13:49:21.143125 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:49:21.143095 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa895b35_c37e_4069_ba0a_306abd972b90.slice/crio-03cecf5b3b3b3f26acd2729d4b1e68efc396e3a82d02ec3f65aae136f89fdd07 WatchSource:0}: Error finding container 03cecf5b3b3b3f26acd2729d4b1e68efc396e3a82d02ec3f65aae136f89fdd07: Status 404 returned error can't find the container with id 03cecf5b3b3b3f26acd2729d4b1e68efc396e3a82d02ec3f65aae136f89fdd07 Jul 17 13:49:21.145019 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:21.145004 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 13:49:21.750383 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:21.750348 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" event={"ID":"aa895b35-c37e-4069-ba0a-306abd972b90","Type":"ContainerStarted","Data":"03cecf5b3b3b3f26acd2729d4b1e68efc396e3a82d02ec3f65aae136f89fdd07"} Jul 17 13:49:28.786192 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:28.786152 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" event={"ID":"aa895b35-c37e-4069-ba0a-306abd972b90","Type":"ContainerStarted","Data":"031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2"} Jul 17 13:49:28.786192 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:28.786199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" event={"ID":"aa895b35-c37e-4069-ba0a-306abd972b90","Type":"ContainerStarted","Data":"f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033"} Jul 17 13:49:28.802108 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:28.802059 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" podStartSLOduration=2.142062941 podStartE2EDuration="8.8020432s" totalImagesPullingTime="6.659980259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 13:49:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:28.799966794 +0000 UTC m=+1810.878080462" watchObservedRunningTime="2026-07-17 13:49:28.8020432 +0000 UTC m=+1810.880156864" Jul 17 13:49:37.828758 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:37.828721 2560 generic.go:356] "Generic (PLEG): container finished" podID="aa895b35-c37e-4069-ba0a-306abd972b90" containerID="f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033" exitCode=0 Jul 17 13:49:37.829320 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:37.828799 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" event={"ID":"aa895b35-c37e-4069-ba0a-306abd972b90","Type":"ContainerDied","Data":"f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033"} Jul 17 13:49:37.829320 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:37.829235 2560 scope.go:117] "RemoveContainer" containerID="f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033" Jul 17 13:49:38.284454 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:38.284423 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gbbsf_must-gather-5cn4l_aa895b35-c37e-4069-ba0a-306abd972b90/gather/0.log" Jul 17 13:49:38.627558 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:38.627478 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:38.779036 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:38.779005 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:38.835902 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:38.835867 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:41.378021 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:41.377991 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9b6c9_b91e071d-432f-4a3d-8f62-f18403981b2d/global-pull-secret-syncer/0.log" Jul 17 13:49:41.476366 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:41.476336 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" Jul 17 13:49:41.600842 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:41.600813 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-189.ec2.internal_068abd2681734d05200c5d2715b73db9/haproxy/0.log" Jul 17 13:49:43.600954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.600892 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-gbbsf/must-gather-5cn4l"] Jul 17 13:49:43.601750 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.601708 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" containerName="copy" containerID="cri-o://031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2" gracePeriod=2 Jul 17 13:49:43.602548 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.602495 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-gbbsf/must-gather-5cn4l"] Jul 17 13:49:43.603925 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:49:43.603881 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-5cn4l\" is forbidden: User \"system:node:ip-10-0-142-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gbbsf\": no relationship found between node 'ip-10-0-142-189.ec2.internal' and this object" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:43.832739 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.832713 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gbbsf_must-gather-5cn4l_aa895b35-c37e-4069-ba0a-306abd972b90/copy/0.log" Jul 17 13:49:43.834239 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.834222 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:43.836369 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:49:43.836345 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-5cn4l\" is forbidden: User \"system:node:ip-10-0-142-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gbbsf\": no relationship found between node 'ip-10-0-142-189.ec2.internal' and this object" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:43.855896 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.855832 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-gbbsf_must-gather-5cn4l_aa895b35-c37e-4069-ba0a-306abd972b90/copy/0.log" Jul 17 13:49:43.856186 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.856166 2560 generic.go:356] "Generic (PLEG): container finished" podID="aa895b35-c37e-4069-ba0a-306abd972b90" containerID="031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2" exitCode=143 Jul 17 13:49:43.856273 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.856219 2560 scope.go:117] "RemoveContainer" containerID="031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2" Jul 17 13:49:43.856316 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.856218 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:43.858539 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:49:43.858513 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-5cn4l\" is forbidden: User \"system:node:ip-10-0-142-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gbbsf\": no relationship found between node 'ip-10-0-142-189.ec2.internal' and this object" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:43.863631 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.863614 2560 scope.go:117] "RemoveContainer" containerID="f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033" Jul 17 13:49:43.875678 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.875655 2560 scope.go:117] "RemoveContainer" containerID="031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2" Jul 17 13:49:43.875986 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:49:43.875959 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2\": container with ID starting with 031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2 not found: ID does not exist" containerID="031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2" Jul 17 13:49:43.876065 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.875994 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2"} err="failed to get container status \"031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2\": rpc error: code = NotFound desc = could not find container \"031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2\": container with ID starting with 031573c634c618d5b4dc736a7edcdff81e4edc03d18894881051b9e416d478c2 not found: ID does not exist" Jul 17 13:49:43.876065 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.876015 2560 scope.go:117] "RemoveContainer" containerID="f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033" Jul 17 13:49:43.876308 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:49:43.876290 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033\": container with ID starting with f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033 not found: ID does not exist" containerID="f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033" Jul 17 13:49:43.876349 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.876312 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033"} err="failed to get container status \"f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033\": rpc error: code = NotFound desc = could not find container \"f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033\": container with ID starting with f1c6a8dc70978798137bdb2a1885742dc0971f562a65ff6505ac92568332a033 not found: ID does not exist" Jul 17 13:49:43.905207 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.905172 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl9g5\" (UniqueName: \"kubernetes.io/projected/aa895b35-c37e-4069-ba0a-306abd972b90-kube-api-access-pl9g5\") pod \"aa895b35-c37e-4069-ba0a-306abd972b90\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " Jul 17 13:49:43.905382 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.905278 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa895b35-c37e-4069-ba0a-306abd972b90-must-gather-output\") pod \"aa895b35-c37e-4069-ba0a-306abd972b90\" (UID: \"aa895b35-c37e-4069-ba0a-306abd972b90\") " Jul 17 13:49:43.905631 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.905610 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa895b35-c37e-4069-ba0a-306abd972b90-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "aa895b35-c37e-4069-ba0a-306abd972b90" (UID: "aa895b35-c37e-4069-ba0a-306abd972b90"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 13:49:43.907285 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.907263 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa895b35-c37e-4069-ba0a-306abd972b90-kube-api-access-pl9g5" (OuterVolumeSpecName: "kube-api-access-pl9g5") pod "aa895b35-c37e-4069-ba0a-306abd972b90" (UID: "aa895b35-c37e-4069-ba0a-306abd972b90"). InnerVolumeSpecName "kube-api-access-pl9g5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 13:49:43.999646 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:43.999608 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:44.005899 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:44.005864 2560 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa895b35-c37e-4069-ba0a-306abd972b90-must-gather-output\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:49:44.005899 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:44.005897 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl9g5\" (UniqueName: \"kubernetes.io/projected/aa895b35-c37e-4069-ba0a-306abd972b90-kube-api-access-pl9g5\") on node \"ip-10-0-142-189.ec2.internal\" DevicePath \"\"" Jul 17 13:49:44.097657 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:44.097623 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:49:44.166630 ip-10-0-142-189 kubenswrapper[2560]: E0717 13:49:44.166599 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-5cn4l\" is forbidden: User \"system:node:ip-10-0-142-189.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-gbbsf\": no relationship found between node 'ip-10-0-142-189.ec2.internal' and this object" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" pod="openshift-must-gather-gbbsf/must-gather-5cn4l" Jul 17 13:49:45.100866 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.100819 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" path="/var/lib/kubelet/pods/aa895b35-c37e-4069-ba0a-306abd972b90/volumes" Jul 17 13:49:45.316212 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.316180 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_103c6d6f-a6a4-45f1-a96f-ac72496052cc/alertmanager/0.log" Jul 17 13:49:45.346425 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.346394 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_103c6d6f-a6a4-45f1-a96f-ac72496052cc/config-reloader/0.log" Jul 17 13:49:45.376411 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.376336 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_103c6d6f-a6a4-45f1-a96f-ac72496052cc/kube-rbac-proxy-web/0.log" Jul 17 13:49:45.399294 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.399261 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_103c6d6f-a6a4-45f1-a96f-ac72496052cc/kube-rbac-proxy/0.log" Jul 17 13:49:45.426845 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.426799 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_103c6d6f-a6a4-45f1-a96f-ac72496052cc/kube-rbac-proxy-metric/0.log" Jul 17 13:49:45.450805 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.450781 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_103c6d6f-a6a4-45f1-a96f-ac72496052cc/prom-label-proxy/0.log" Jul 17 13:49:45.471890 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.471870 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_103c6d6f-a6a4-45f1-a96f-ac72496052cc/init-config-reloader/0.log" Jul 17 13:49:45.543107 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.543074 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-s9j7s_bbd3654e-332f-40c0-aed6-026887beae62/kube-state-metrics/0.log" Jul 17 13:49:45.569506 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.569476 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-s9j7s_bbd3654e-332f-40c0-aed6-026887beae62/kube-rbac-proxy-main/0.log" Jul 17 13:49:45.594954 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.594929 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-s9j7s_bbd3654e-332f-40c0-aed6-026887beae62/kube-rbac-proxy-self/0.log" Jul 17 13:49:45.632729 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.632658 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-76599cfbd6-w7cnp_20685541-c9ad-4c7c-bc06-21371aff740f/metrics-server/0.log" Jul 17 13:49:45.660424 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.660391 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-pr6wb_be85d966-ec82-4885-932f-143a63405dbd/monitoring-plugin/0.log" Jul 17 13:49:45.762991 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.762964 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8c9sg_dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31/node-exporter/0.log" Jul 17 13:49:45.785329 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.785303 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8c9sg_dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31/kube-rbac-proxy/0.log" Jul 17 13:49:45.810923 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.810896 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8c9sg_dbd3e54f-3fa8-4e1d-8a8f-b815f75d8b31/init-textfile/0.log" Jul 17 13:49:45.911451 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.911383 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vf52q_9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2/kube-rbac-proxy-main/0.log" Jul 17 13:49:45.932040 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.932015 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vf52q_9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2/kube-rbac-proxy-self/0.log" Jul 17 13:49:45.954650 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.954610 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-vf52q_9d4bde7a-a3bd-423e-8ba0-b114f4d1a2a2/openshift-state-metrics/0.log" Jul 17 13:49:45.998307 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:45.998280 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf1f11cb-a084-4d66-bbaf-ef3d52061ded/prometheus/0.log" Jul 17 13:49:46.025990 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.025957 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf1f11cb-a084-4d66-bbaf-ef3d52061ded/config-reloader/0.log" Jul 17 13:49:46.056050 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.056023 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf1f11cb-a084-4d66-bbaf-ef3d52061ded/thanos-sidecar/0.log" Jul 17 13:49:46.083774 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.083750 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf1f11cb-a084-4d66-bbaf-ef3d52061ded/kube-rbac-proxy-web/0.log" Jul 17 13:49:46.109418 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.109391 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf1f11cb-a084-4d66-bbaf-ef3d52061ded/kube-rbac-proxy/0.log" Jul 17 13:49:46.130472 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.130449 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf1f11cb-a084-4d66-bbaf-ef3d52061ded/kube-rbac-proxy-thanos/0.log" Jul 17 13:49:46.158031 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.158009 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_cf1f11cb-a084-4d66-bbaf-ef3d52061ded/init-config-reloader/0.log" Jul 17 13:49:46.191489 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.191460 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jlqcj_216a0da8-fb64-403c-8291-54f620df8fa8/prometheus-operator/0.log" Jul 17 13:49:46.213553 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.213518 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-jlqcj_216a0da8-fb64-403c-8291-54f620df8fa8/kube-rbac-proxy/0.log" Jul 17 13:49:46.239905 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.239883 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-n9qkq_64e794d3-30e3-4447-9f4b-6516c3ed9af3/prometheus-operator-admission-webhook/0.log" Jul 17 13:49:46.277294 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.277273 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555969c789-bscs4_6a664b5a-72aa-4fc3-8738-93d0a51744cc/telemeter-client/0.log" Jul 17 13:49:46.297827 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.297803 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555969c789-bscs4_6a664b5a-72aa-4fc3-8738-93d0a51744cc/reload/0.log" Jul 17 13:49:46.324994 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.324972 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-555969c789-bscs4_6a664b5a-72aa-4fc3-8738-93d0a51744cc/kube-rbac-proxy/0.log" Jul 17 13:49:46.353352 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.353328 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c997d6f94-bf7np_e110e86d-d307-469c-8ce9-1c369156ac8d/thanos-query/0.log" Jul 17 13:49:46.375111 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.375086 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c997d6f94-bf7np_e110e86d-d307-469c-8ce9-1c369156ac8d/kube-rbac-proxy-web/0.log" Jul 17 13:49:46.398164 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.398140 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c997d6f94-bf7np_e110e86d-d307-469c-8ce9-1c369156ac8d/kube-rbac-proxy/0.log" Jul 17 13:49:46.428121 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.428096 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c997d6f94-bf7np_e110e86d-d307-469c-8ce9-1c369156ac8d/prom-label-proxy/0.log" Jul 17 13:49:46.467075 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.467005 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c997d6f94-bf7np_e110e86d-d307-469c-8ce9-1c369156ac8d/kube-rbac-proxy-rules/0.log" Jul 17 13:49:46.501180 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:46.501159 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5c997d6f94-bf7np_e110e86d-d307-469c-8ce9-1c369156ac8d/kube-rbac-proxy-metrics/0.log" Jul 17 13:49:48.765085 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.765048 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq"] Jul 17 13:49:48.765484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.765445 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" containerName="copy" Jul 17 13:49:48.765484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.765458 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" containerName="copy" Jul 17 13:49:48.765484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.765478 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" containerName="gather" Jul 17 13:49:48.765484 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.765484 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" containerName="gather" Jul 17 13:49:48.765620 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.765534 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" containerName="copy" Jul 17 13:49:48.765620 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.765545 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa895b35-c37e-4069-ba0a-306abd972b90" containerName="gather" Jul 17 13:49:48.768776 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.768755 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.771366 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.771348 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n4gdz\"/\"kube-root-ca.crt\"" Jul 17 13:49:48.771485 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.771467 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-n4gdz\"/\"default-dockercfg-ztd6p\"" Jul 17 13:49:48.771542 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.771530 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-n4gdz\"/\"openshift-service-ca.crt\"" Jul 17 13:49:48.778581 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.778555 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq"] Jul 17 13:49:48.845397 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.845359 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-sys\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.845397 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.845398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bfl7\" (UniqueName: \"kubernetes.io/projected/1aa92dd4-841f-466e-9536-9ca80d35a823-kube-api-access-5bfl7\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.845609 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.845430 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-podres\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.845609 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.845497 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-lib-modules\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.845609 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.845559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-proc\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946543 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-sys\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946543 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946544 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bfl7\" (UniqueName: \"kubernetes.io/projected/1aa92dd4-841f-466e-9536-9ca80d35a823-kube-api-access-5bfl7\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946573 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-podres\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-lib-modules\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946626 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-sys\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946651 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-proc\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946721 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-podres\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946755 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-lib-modules\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.946804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.946758 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1aa92dd4-841f-466e-9536-9ca80d35a823-proc\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:48.954167 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:48.954150 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bfl7\" (UniqueName: \"kubernetes.io/projected/1aa92dd4-841f-466e-9536-9ca80d35a823-kube-api-access-5bfl7\") pod \"perf-node-gather-daemonset-qghnq\" (UID: \"1aa92dd4-841f-466e-9536-9ca80d35a823\") " pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:49.079347 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.079253 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:49.209073 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.209040 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq"] Jul 17 13:49:49.212371 ip-10-0-142-189 kubenswrapper[2560]: W0717 13:49:49.212340 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1aa92dd4_841f_466e_9536_9ca80d35a823.slice/crio-2c1a12e8f447dd7d459187890debc10b7d9556b88c608d4138f5f84ce7195d68 WatchSource:0}: Error finding container 2c1a12e8f447dd7d459187890debc10b7d9556b88c608d4138f5f84ce7195d68: Status 404 returned error can't find the container with id 2c1a12e8f447dd7d459187890debc10b7d9556b88c608d4138f5f84ce7195d68 Jul 17 13:49:49.773491 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.773458 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9j845_e65e5d8d-b568-4dea-a911-762096915096/dns/0.log" Jul 17 13:49:49.793172 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.793144 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9j845_e65e5d8d-b568-4dea-a911-762096915096/kube-rbac-proxy/0.log" Jul 17 13:49:49.881607 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.881569 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" event={"ID":"1aa92dd4-841f-466e-9536-9ca80d35a823","Type":"ContainerStarted","Data":"e9bab1abd273743590e67369fa223675ff088fbd855942aaa7270331e007e10a"} Jul 17 13:49:49.881799 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.881621 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" event={"ID":"1aa92dd4-841f-466e-9536-9ca80d35a823","Type":"ContainerStarted","Data":"2c1a12e8f447dd7d459187890debc10b7d9556b88c608d4138f5f84ce7195d68"} Jul 17 13:49:49.881799 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.881675 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:49.896346 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.896299 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" podStartSLOduration=1.896284646 podStartE2EDuration="1.896284646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 13:49:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 13:49:49.895617555 +0000 UTC m=+1831.973731221" watchObservedRunningTime="2026-07-17 13:49:49.896284646 +0000 UTC m=+1831.974398309" Jul 17 13:49:49.901463 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:49.901447 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-52ncl_a463f487-d248-42f7-8745-f3b1e6869b9f/dns-node-resolver/0.log" Jul 17 13:49:50.365993 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:50.365964 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-twxnh_28f66006-4ac8-4413-bd8c-e355c9cb4c8c/node-ca/0.log" Jul 17 13:49:51.413639 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:51.413612 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xlfpb_f927fd7f-9e9a-4af5-8fd7-76c057df969d/serve-healthcheck-canary/0.log" Jul 17 13:49:51.741575 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:51.741549 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5j8xf_3def5554-1674-4cfe-9da7-16f997510485/kube-rbac-proxy/0.log" Jul 17 13:49:51.760284 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:51.760266 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5j8xf_3def5554-1674-4cfe-9da7-16f997510485/exporter/0.log" Jul 17 13:49:51.778675 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:51.778661 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5j8xf_3def5554-1674-4cfe-9da7-16f997510485/extractor/0.log" Jul 17 13:49:55.897271 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:55.897236 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-n4gdz/perf-node-gather-daemonset-qghnq" Jul 17 13:49:56.970219 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:56.970194 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lv575_99db032f-ab0a-405e-9482-74bd20a526f1/kube-multus-additional-cni-plugins/0.log" Jul 17 13:49:56.992231 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:56.992198 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lv575_99db032f-ab0a-405e-9482-74bd20a526f1/egress-router-binary-copy/0.log" Jul 17 13:49:57.014268 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.014239 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lv575_99db032f-ab0a-405e-9482-74bd20a526f1/cni-plugins/0.log" Jul 17 13:49:57.037095 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.037075 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lv575_99db032f-ab0a-405e-9482-74bd20a526f1/bond-cni-plugin/0.log" Jul 17 13:49:57.059530 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.059510 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lv575_99db032f-ab0a-405e-9482-74bd20a526f1/routeoverride-cni/0.log" Jul 17 13:49:57.080198 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.080177 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lv575_99db032f-ab0a-405e-9482-74bd20a526f1/whereabouts-cni-bincopy/0.log" Jul 17 13:49:57.101436 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.101420 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lv575_99db032f-ab0a-405e-9482-74bd20a526f1/whereabouts-cni/0.log" Jul 17 13:49:57.440786 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.440751 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-t2xx7_7d1bb89e-f6cc-4f03-a214-4ba879a513d0/kube-multus/0.log" Jul 17 13:49:57.527295 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.527268 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pfz9l_1992cdcc-16ba-4ebe-a0a8-b382deed8fcb/network-metrics-daemon/0.log" Jul 17 13:49:57.548808 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:57.548784 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pfz9l_1992cdcc-16ba-4ebe-a0a8-b382deed8fcb/kube-rbac-proxy/0.log" Jul 17 13:49:58.788930 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.788885 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/ovn-controller/0.log" Jul 17 13:49:58.809609 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.809581 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/ovn-acl-logging/0.log" Jul 17 13:49:58.829375 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.829350 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/kube-rbac-proxy-node/0.log" Jul 17 13:49:58.848534 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.848507 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/kube-rbac-proxy-ovn-metrics/0.log" Jul 17 13:49:58.872392 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.872367 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/northd/0.log" Jul 17 13:49:58.919630 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.919604 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/nbdb/0.log" Jul 17 13:49:58.940804 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.940772 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/sbdb/0.log" Jul 17 13:49:58.980638 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:58.980616 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s478r_e3b450f3-ffc7-4ea6-8283-7772fae77cbc/ovnkube-controller/0.log" Jul 17 13:49:59.990522 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:49:59.990493 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-8nhwn_2441c8b2-c301-47f9-96f2-eb0049a7dda3/network-check-target-container/0.log" Jul 17 13:50:00.873688 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:50:00.873659 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-wxgng_ff48e2f8-23d2-41e0-ad4e-ca761c116732/iptables-alerter/0.log" Jul 17 13:50:01.480787 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:50:01.480742 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-m5nr4_6f5bfaeb-5c9c-4b0b-98e5-c27acb371162/tuned/0.log" Jul 17 13:50:03.222666 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:50:03.222635 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-44d5g_8180fada-f9bd-41d0-943e-70df65e3e2cd/konnectivity-agent/0.log" limit=1024 Jul 17 13:50:04.619040 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:50:04.619008 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pkjlv_a625f32a-cfe2-4918-9abe-1cda781faf78/csi-driver/0.log" Jul 17 13:50:04.641030 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:50:04.641008 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pkjlv_a625f32a-cfe2-4918-9abe-1cda781faf78/csi-node-driver-registrar/0.log" Jul 17 13:50:04.659577 ip-10-0-142-189 kubenswrapper[2560]: I0717 13:50:04.659560 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-pkjlv_a625f32a-cfe2-4918-9abe-1cda781faf78/csi-liveness-probe/0.log"