Jul 28 17:17:21.668154 ip-10-0-129-206 systemd[1]: Starting Kubernetes Kubelet... Jul 28 17:17:22.069024 ip-10-0-129-206 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:17:22.069024 ip-10-0-129-206 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 17:17:22.069024 ip-10-0-129-206 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:17:22.069024 ip-10-0-129-206 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 17:17:22.069024 ip-10-0-129-206 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:17:22.070230 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.070164 2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 17:17:22.076902 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076888 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:22.076902 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076902 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076906 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076910 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076913 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076917 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076920 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076923 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076926 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076929 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076932 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076935 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076939 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076942 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076944 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076947 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076950 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076952 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076955 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076958 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:22.076967 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076961 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076963 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076966 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076970 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076973 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076975 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076978 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076981 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076984 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076988 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076992 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076994 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.076997 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077000 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077002 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077005 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077007 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077010 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077013 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077015 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:22.077445 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077018 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077020 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077023 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077025 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077029 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077032 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077035 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077038 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077041 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077044 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077047 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077050 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077052 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077055 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077058 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077061 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077064 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077067 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077069 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:22.077946 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077072 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077074 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077077 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077079 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077082 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077084 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077087 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077090 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077093 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077096 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077098 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077101 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077107 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077109 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077112 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077114 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077117 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077120 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077122 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077125 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:22.078414 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077129 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077132 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077135 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077138 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077140 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077143 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077145 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077148 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077151 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077153 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077156 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077159 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077161 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077164 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077166 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077169 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077171 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077174 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077176 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077179 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:22.079349 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077906 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077913 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077917 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077919 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077922 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077932 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077934 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077937 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077940 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077942 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077945 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077948 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077951 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077954 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077956 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077961 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077966 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077968 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077972 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:22.079845 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077975 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077978 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077982 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077985 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077988 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077991 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077993 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077996 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.077999 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078001 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078004 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078006 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078009 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078011 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078014 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078016 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078019 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078021 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078024 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:22.080361 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078027 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078030 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078032 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078035 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078038 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078041 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078044 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078046 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078049 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078052 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078054 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078057 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078059 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078062 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078064 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078067 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078069 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078072 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078074 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:22.080823 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078077 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078079 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078082 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078084 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078087 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078089 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078107 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078110 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078113 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078117 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078119 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078122 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078125 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078128 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078131 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078134 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078136 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078139 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078142 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078145 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:22.081287 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078147 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078150 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078152 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078155 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078157 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078160 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078163 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078167 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078171 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078174 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078177 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078179 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078182 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078184 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078187 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078190 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078192 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078194 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078197 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078199 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:22.081824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078202 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078204 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078576 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078580 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078583 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078586 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078589 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078592 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078595 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078598 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078602 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078606 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078609 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078611 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078614 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078617 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078619 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078622 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078625 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:22.082316 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078627 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078630 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078632 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078636 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078640 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078643 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078646 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078648 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078651 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078654 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078657 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078659 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078662 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078664 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078667 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078669 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078672 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078674 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078678 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:22.082800 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078681 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078685 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078688 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078692 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078695 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078697 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078700 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078702 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078705 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078707 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078710 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078712 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078715 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078717 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078720 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078722 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078725 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078727 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078730 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:22.083277 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078732 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078749 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078753 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078756 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078759 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078762 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078764 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078767 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078770 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078772 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078775 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078778 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078780 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078783 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078786 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078789 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078792 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078795 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078798 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078801 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:22.083736 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078804 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078806 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078808 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078811 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078813 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078816 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078819 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078822 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078824 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078827 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078829 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078832 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078837 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078839 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078842 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078844 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078846 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078849 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078851 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078854 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:22.084218 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078856 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078859 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078861 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.078864 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078946 2554 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078956 2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078965 2554 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078982 2554 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078987 2554 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078990 2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078994 2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.078998 2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079002 2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079005 2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079008 2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079011 2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079014 2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079017 2554 flags.go:64] FLAG: --cgroup-root="" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079020 2554 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079023 2554 flags.go:64] FLAG: --client-ca-file="" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079026 2554 flags.go:64] FLAG: --cloud-provider="external" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079029 2554 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079033 2554 flags.go:64] FLAG: --cluster-domain="" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079036 2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 17:17:22.084690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079041 2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079044 2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079048 2554 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079051 2554 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079055 2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079058 2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079061 2554 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079064 2554 flags.go:64] FLAG: --contention-profiling="false" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079070 2554 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079072 2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079076 2554 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079078 2554 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079082 2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079085 2554 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079088 2554 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079091 2554 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079094 2554 flags.go:64] FLAG: --enable-server="true" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079097 2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079101 2554 flags.go:64] FLAG: --event-burst="100" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079104 2554 flags.go:64] FLAG: --event-qps="50" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079107 2554 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079110 2554 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079113 2554 flags.go:64] FLAG: --eviction-hard="" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079116 2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 17:17:22.085324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079119 2554 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079122 2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079125 2554 flags.go:64] FLAG: --eviction-soft="" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079128 2554 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079131 2554 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079133 2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079136 2554 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079140 2554 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079144 2554 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079147 2554 flags.go:64] FLAG: --feature-gates="" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079150 2554 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079153 2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079156 2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079159 2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079162 2554 flags.go:64] FLAG: --healthz-port="10248" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079165 2554 flags.go:64] FLAG: --help="false" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079169 2554 flags.go:64] FLAG: --hostname-override="ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079172 2554 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079175 2554 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079178 2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079181 2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079185 2554 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079188 2554 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 17:17:22.085948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079191 2554 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079194 2554 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079196 2554 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079200 2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079203 2554 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079205 2554 flags.go:64] FLAG: --kube-reserved="" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079208 2554 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079211 2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079215 2554 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079217 2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079220 2554 flags.go:64] FLAG: --lock-file="" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079223 2554 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079226 2554 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079229 2554 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079234 2554 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079237 2554 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079239 2554 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079244 2554 flags.go:64] FLAG: --logging-format="text" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079246 2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079250 2554 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079252 2554 flags.go:64] FLAG: --manifest-url="" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079255 2554 flags.go:64] FLAG: --manifest-url-header="" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079259 2554 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079262 2554 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079266 2554 flags.go:64] FLAG: --max-pods="110" Jul 28 17:17:22.086703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079270 2554 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079273 2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079276 2554 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079279 2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079281 2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079284 2554 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079287 2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079294 2554 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079297 2554 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079300 2554 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079303 2554 flags.go:64] FLAG: --pod-cidr="" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079306 2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079311 2554 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079314 2554 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079318 2554 flags.go:64] FLAG: --pods-per-core="0" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079321 2554 flags.go:64] FLAG: --port="10250" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079324 2554 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079327 2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08a8a77e76c1db24c" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079330 2554 flags.go:64] FLAG: --qos-reserved="" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079333 2554 flags.go:64] FLAG: --read-only-port="10255" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079336 2554 flags.go:64] FLAG: --register-node="true" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079339 2554 flags.go:64] FLAG: --register-with-taints="" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079343 2554 flags.go:64] FLAG: --registry-burst="10" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079345 2554 flags.go:64] FLAG: --registry-qps="5" Jul 28 17:17:22.087359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079349 2554 flags.go:64] FLAG: --reserved-cpus="" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079352 2554 flags.go:64] FLAG: --reserved-memory="" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079356 2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079359 2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079362 2554 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079365 2554 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079368 2554 flags.go:64] FLAG: --runonce="false" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079370 2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079374 2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079378 2554 flags.go:64] FLAG: --seccomp-default="false" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079381 2554 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079384 2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079398 2554 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079402 2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079405 2554 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079408 2554 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079411 2554 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079413 2554 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079416 2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079419 2554 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079423 2554 flags.go:64] FLAG: --system-cgroups="" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079426 2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079431 2554 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079434 2554 flags.go:64] FLAG: --tls-cert-file="" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079436 2554 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 17:17:22.088012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079444 2554 flags.go:64] FLAG: --tls-min-version="" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079447 2554 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079450 2554 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079453 2554 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079455 2554 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079458 2554 flags.go:64] FLAG: --v="2" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079462 2554 flags.go:64] FLAG: --version="false" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079467 2554 flags.go:64] FLAG: --vmodule="" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079471 2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.079474 2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079585 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079589 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079592 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079595 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079598 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079600 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079604 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079607 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079609 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079612 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079615 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079618 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:22.088609 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079621 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079623 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079626 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079628 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079631 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079634 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079637 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079640 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079642 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079645 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079647 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079650 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079652 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079655 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079657 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079660 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079663 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079667 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079669 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079672 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:22.089160 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079675 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079677 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079680 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079683 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079685 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079688 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079692 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079694 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079697 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079700 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079702 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079705 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079707 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079710 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079712 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079715 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079718 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079720 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079723 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:22.089675 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079726 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079729 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079733 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079737 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079740 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079743 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079746 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079748 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079751 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079753 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079757 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079760 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079763 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079765 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079768 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079770 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079773 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079776 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079778 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:22.090147 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079783 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079786 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079788 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079791 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079793 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079796 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079798 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079801 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079804 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079806 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079809 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079812 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079816 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079820 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079825 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079828 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079831 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079834 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079837 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:22.090611 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079840 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079843 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079845 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079848 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079853 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079855 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079858 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079861 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079863 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.079866 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.080446 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.088372 2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.088386 2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.089178 2554 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.089186 2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 17:17:22.091078 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089245 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089250 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089253 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089256 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089260 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089263 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089265 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089268 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089270 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089275 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089278 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089281 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089283 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089286 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089289 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089291 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089294 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089297 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089301 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:22.091458 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089305 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089308 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089311 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089315 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089317 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089321 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089323 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089326 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089328 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089331 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089333 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089337 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089340 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089342 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089345 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089348 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089351 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089355 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089358 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:22.091928 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089361 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089364 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089367 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089370 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089373 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089376 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089378 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089380 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089384 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089387 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089389 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089391 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089394 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089396 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089399 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089402 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089404 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089407 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089410 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089412 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:22.092383 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089415 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089417 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089420 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089422 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089425 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089428 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089430 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089433 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089435 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089438 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089440 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089443 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089445 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089448 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089450 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089453 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089456 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089459 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089461 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089464 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:22.092854 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089466 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089469 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089472 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089474 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089476 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089479 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089481 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089484 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089486 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089489 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089491 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089494 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089496 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089499 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089501 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089504 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089506 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089509 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089511 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089514 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:22.093442 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089517 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.089522 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089633 2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089638 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089641 2554 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089644 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089646 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089649 2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089652 2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089654 2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089657 2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089659 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089662 2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089664 2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089667 2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:17:22.093948 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089670 2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089672 2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089675 2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089678 2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089681 2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089685 2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089690 2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089694 2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089696 2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089699 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089703 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089705 2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089708 2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089711 2554 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089713 2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089716 2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089718 2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089721 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089723 2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:17:22.094362 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089726 2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089728 2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089731 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089733 2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089736 2554 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089738 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089741 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089743 2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089746 2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089748 2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089751 2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089753 2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089756 2554 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089759 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089761 2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089764 2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089766 2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089769 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089771 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089774 2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:17:22.094815 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089777 2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089781 2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089784 2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089787 2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089791 2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089794 2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089796 2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089799 2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089801 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089804 2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089807 2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089809 2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089812 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089814 2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089817 2554 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089819 2554 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089822 2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089824 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089827 2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089829 2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:17:22.095313 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089832 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089834 2554 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089837 2554 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089839 2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089842 2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089844 2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089847 2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089850 2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089852 2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089855 2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089857 2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089860 2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089895 2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089899 2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089903 2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089906 2554 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089910 2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089913 2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089916 2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089918 2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:17:22.095827 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089921 2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089924 2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089927 2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089930 2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089933 2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089936 2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:22.089938 2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.089943 2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.090048 2554 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.094026 2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.095317 2554 server.go:1019] "Starting client certificate rotation" Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.095418 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:17:22.096370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.095449 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:17:22.097739 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.097726 2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:17:22.100765 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.100752 2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:17:22.113350 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.113327 2554 log.go:25] "Validated CRI v1 runtime API" Jul 28 17:17:22.116611 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.116591 2554 log.go:25] "Validated CRI v1 image API" Jul 28 17:17:22.117435 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.117417 2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 17:17:22.120076 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.120059 2554 fs.go:135] Filesystem UUIDs: map[175744c4-da5b-4efa-8142-9b525203be00:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c62f3ae6-2989-46e8-9c1f-f287d7ec5eed:/dev/nvme0n1p3] Jul 28 17:17:22.120142 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.120076 2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 17:17:22.125065 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.125042 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:17:22.126231 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.126122 2554 manager.go:216] Machine: {Timestamp:2026-07-28 17:17:22.124680993 +0000 UTC m=+0.342722448 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099591 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2b84379550becdbc355a27b1fd62ea SystemUUID:ec2b8437-9550-becd-bc35-5a27b1fd62ea BootID:0a8f43bc-57d2-4d9b-896c-913a59c5055c Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:93:88:eb:e5:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:93:88:eb:e5:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:40:78:f1:ba:77 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 17:17:22.126231 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.126222 2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 17:17:22.126393 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.126352 2554 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 17:17:22.129522 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.129496 2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 17:17:22.129646 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.129524 2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-206.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 17:17:22.130355 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.130344 2554 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 17:17:22.130393 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.130357 2554 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 17:17:22.130393 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.130364 2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:17:22.130990 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.130979 2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:17:22.131684 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.131674 2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 17:17:22.133397 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133385 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:17:22.133517 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133507 2554 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 17:17:22.133553 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133538 2554 kubelet.go:486] "Attempting to sync node with API server" Jul 28 17:17:22.133553 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133550 2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 17:17:22.133607 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133563 2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 17:17:22.133607 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133573 2554 kubelet.go:398] "Adding apiserver pod source" Jul 28 17:17:22.133607 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133581 2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 17:17:22.133930 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133919 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:17:22.133970 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.133939 2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:17:22.136156 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.136140 2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 17:17:22.137851 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.137836 2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 17:17:22.137927 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.137865 2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 17:17:22.139418 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139406 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 17:17:22.139451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139423 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 17:17:22.139451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139430 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 17:17:22.139451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139435 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 17:17:22.139451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139444 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 17:17:22.139451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139451 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 17:17:22.139583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139459 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 17:17:22.139583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139464 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 17:17:22.139583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139470 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 17:17:22.139583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139476 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 17:17:22.139583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139490 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 17:17:22.139583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.139495 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 17:17:22.140975 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.140964 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 17:17:22.141013 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.140975 2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 17:17:22.143857 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.143843 2554 server.go:1268] "Started kubelet" Jul 28 17:17:22.144017 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.143958 2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 17:17:22.144105 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.144003 2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 17:17:22.144105 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.144094 2554 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 17:17:22.144618 ip-10-0-129-206 systemd[1]: Started Kubernetes Kubelet. Jul 28 17:17:22.146000 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.145981 2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 17:17:22.146350 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.146336 2554 server.go:310] "Adding debug handlers to kubelet server" Jul 28 17:17:22.146482 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.146459 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 17:17:22.146482 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.146467 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 17:17:22.146609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.146527 2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-206.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 17:17:22.149131 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.148239 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-206.ec2.internal.18c683a3df0944ed default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-206.ec2.internal,UID:ip-10-0-129-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-206.ec2.internal,},FirstTimestamp:2026-07-28 17:17:22.143814893 +0000 UTC m=+0.361856346,LastTimestamp:2026-07-28 17:17:22.143814893 +0000 UTC m=+0.361856346,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-206.ec2.internal,}" Jul 28 17:17:22.149734 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.149706 2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 17:17:22.152332 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.152312 2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 17:17:22.152332 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.152319 2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 17:17:22.152591 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.152568 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.152924 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.152905 2554 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 17:17:22.152924 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.152925 2554 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 17:17:22.153031 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.152908 2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 17:17:22.153031 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.153012 2554 reconstruct.go:97] "Volume reconstruction finished" Jul 28 17:17:22.153031 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.153020 2554 reconciler.go:29] "Reconciler: start to sync state" Jul 28 17:17:22.153572 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.153553 2554 factory.go:55] Registering systemd factory Jul 28 17:17:22.153572 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.153574 2554 factory.go:223] Registration of the systemd container factory successfully Jul 28 17:17:22.153992 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.153973 2554 factory.go:153] Registering CRI-O factory Jul 28 17:17:22.154061 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.153999 2554 factory.go:223] Registration of the crio container factory successfully Jul 28 17:17:22.154061 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.154048 2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 17:17:22.154156 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.154069 2554 factory.go:103] Registering Raw factory Jul 28 17:17:22.154156 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.154087 2554 manager.go:1196] Started watching for new ooms in manager Jul 28 17:17:22.154656 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.154619 2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 17:17:22.154773 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.154750 2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 17:17:22.155040 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.155023 2554 manager.go:349] Starting recovery of all containers Jul 28 17:17:22.166372 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.166348 2554 manager.go:354] Recovery completed Jul 28 17:17:22.170450 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.170435 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:22.172189 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.172173 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:22.172268 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.172203 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:22.172268 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.172218 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:22.172765 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.172741 2554 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 17:17:22.172765 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.172753 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:22.172765 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.172757 2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 17:17:22.172955 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.172776 2554 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:17:22.173433 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.173418 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:22.173480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.173444 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:22.173480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.173457 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:22.174626 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.174612 2554 policy_none.go:49] "None policy: Start" Jul 28 17:17:22.174660 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.174635 2554 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 17:17:22.174660 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.174650 2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 17:17:22.175782 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.175771 2554 policy_none.go:47] "Start" Jul 28 17:17:22.184826 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.184752 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-206.ec2.internal.18c683a3e0ba390b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-206.ec2.internal,UID:ip-10-0-129-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-206.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-206.ec2.internal,},FirstTimestamp:2026-07-28 17:17:22.172188939 +0000 UTC m=+0.390230397,LastTimestamp:2026-07-28 17:17:22.172188939 +0000 UTC m=+0.390230397,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-206.ec2.internal,}" Jul 28 17:17:22.189780 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.189765 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qpszx" Jul 28 17:17:22.193387 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.193301 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-206.ec2.internal.18c683a3e0ba8c41 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-206.ec2.internal,UID:ip-10-0-129-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-206.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-206.ec2.internal,},FirstTimestamp:2026-07-28 17:17:22.172210241 +0000 UTC m=+0.390251697,LastTimestamp:2026-07-28 17:17:22.172210241 +0000 UTC m=+0.390251697,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-206.ec2.internal,}" Jul 28 17:17:22.202181 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.202161 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qpszx" Jul 28 17:17:22.203913 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.203762 2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-206.ec2.internal.18c683a3e0bac06c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-206.ec2.internal,UID:ip-10-0-129-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-206.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-206.ec2.internal,},FirstTimestamp:2026-07-28 17:17:22.172223596 +0000 UTC m=+0.390265051,LastTimestamp:2026-07-28 17:17:22.172223596 +0000 UTC m=+0.390265051,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-206.ec2.internal,}" Jul 28 17:17:22.218463 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.218449 2554 manager.go:337] "Starting Device Plugin manager" Jul 28 17:17:22.218519 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.218481 2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 17:17:22.218519 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.218493 2554 server.go:87] "Starting device plugin registration server" Jul 28 17:17:22.218715 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.218702 2554 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 17:17:22.218782 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.218716 2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 17:17:22.218825 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.218808 2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 17:17:22.218923 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.218899 2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 17:17:22.218923 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.218913 2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 17:17:22.219375 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.219355 2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 17:17:22.219443 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.219414 2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.318915 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.318897 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:22.320165 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.319895 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:22.320165 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.319921 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:22.320165 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.319933 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:22.320165 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.319955 2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.330370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.330343 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 17:17:22.331403 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.331377 2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 17:17:22.331473 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.331414 2554 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 17:17:22.331473 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.331431 2554 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 17:17:22.331473 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.331464 2554 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 17:17:22.342771 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.342756 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:22.349696 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.349682 2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.349745 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.349701 2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-206.ec2.internal\": node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.397551 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.397524 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.435041 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.434360 2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal"] Jul 28 17:17:22.435124 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.435073 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:22.436117 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.436101 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:22.436205 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.436131 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:22.436205 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.436146 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:22.437760 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.437745 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:22.437903 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.437870 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.437971 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.437919 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:22.438453 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.438436 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:22.438453 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.438453 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:22.438563 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.438466 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:22.438563 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.438476 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:22.438563 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.438482 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:22.438563 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.438486 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:22.439724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.439710 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.439767 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.439736 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:22.440520 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.440502 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:22.440596 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.440536 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:22.440596 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.440551 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:22.454422 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.454377 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcb67e56a80fff0dbe2625d782081ff2-config\") pod \"kube-apiserver-proxy-ip-10-0-129-206.ec2.internal\" (UID: \"fcb67e56a80fff0dbe2625d782081ff2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.454500 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.454423 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2901d1a7c56239f90dca1ec3946cd97a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal\" (UID: \"2901d1a7c56239f90dca1ec3946cd97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.454500 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.454443 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2901d1a7c56239f90dca1ec3946cd97a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal\" (UID: \"2901d1a7c56239f90dca1ec3946cd97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.464607 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.464587 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-206.ec2.internal\" not found" node="ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.469066 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.469051 2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-206.ec2.internal\" not found" node="ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.497707 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.497682 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.555143 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.555122 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2901d1a7c56239f90dca1ec3946cd97a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal\" (UID: \"2901d1a7c56239f90dca1ec3946cd97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.555203 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.555154 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcb67e56a80fff0dbe2625d782081ff2-config\") pod \"kube-apiserver-proxy-ip-10-0-129-206.ec2.internal\" (UID: \"fcb67e56a80fff0dbe2625d782081ff2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.555203 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.555172 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2901d1a7c56239f90dca1ec3946cd97a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal\" (UID: \"2901d1a7c56239f90dca1ec3946cd97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.555270 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.555216 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fcb67e56a80fff0dbe2625d782081ff2-config\") pod \"kube-apiserver-proxy-ip-10-0-129-206.ec2.internal\" (UID: \"fcb67e56a80fff0dbe2625d782081ff2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.555270 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.555223 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2901d1a7c56239f90dca1ec3946cd97a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal\" (UID: \"2901d1a7c56239f90dca1ec3946cd97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.555270 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.555255 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2901d1a7c56239f90dca1ec3946cd97a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal\" (UID: \"2901d1a7c56239f90dca1ec3946cd97a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.599231 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.599179 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.700381 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.700353 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.767769 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.767752 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.772987 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:22.772972 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" Jul 28 17:17:22.801094 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.801076 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:22.902790 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:22.902732 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.006284 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:23.006263 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.097192 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.097168 2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 17:17:23.097526 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.097287 2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:17:23.107021 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:23.107006 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.206348 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.206283 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:12:22 +0000 UTC" deadline="2028-03-12 05:27:13.390835599 +0000 UTC" Jul 28 17:17:23.206348 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.206316 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14220h9m50.184522359s" Jul 28 17:17:23.207459 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:23.207444 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.261709 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.261689 2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:23.289557 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.289535 2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:17:23.290672 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.290656 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:17:23.290747 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.290690 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:17:23.290747 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.290701 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:17:23.307499 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:23.307476 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.408209 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:23.408183 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.508493 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:23.508461 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.545049 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:23.545011 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcb67e56a80fff0dbe2625d782081ff2.slice/crio-c265cff910b47ef49a7faa460c207a4071bbfab68f3019358442745d1544f80a WatchSource:0}: Error finding container c265cff910b47ef49a7faa460c207a4071bbfab68f3019358442745d1544f80a: Status 404 returned error can't find the container with id c265cff910b47ef49a7faa460c207a4071bbfab68f3019358442745d1544f80a Jul 28 17:17:23.549938 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.549922 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:17:23.572645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.572627 2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:23.609167 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:23.609143 2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-206.ec2.internal\" not found" Jul 28 17:17:23.656244 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.656230 2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:23.753056 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.753029 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" Jul 28 17:17:23.763171 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:23.763149 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2901d1a7c56239f90dca1ec3946cd97a.slice/crio-23915e2ccfaef85c31645f939042c68001cb7a05b7a244f883fc72ee66631cfe WatchSource:0}: Error finding container 23915e2ccfaef85c31645f939042c68001cb7a05b7a244f883fc72ee66631cfe: Status 404 returned error can't find the container with id 23915e2ccfaef85c31645f939042c68001cb7a05b7a244f883fc72ee66631cfe Jul 28 17:17:23.768214 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.768197 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:17:23.769394 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.769379 2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" Jul 28 17:17:23.780091 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:23.780067 2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:17:24.135789 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.135746 2554 apiserver.go:52] "Watching apiserver" Jul 28 17:17:24.145664 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.145643 2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 17:17:24.145979 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.145960 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-dv5bc","kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt","openshift-cluster-node-tuning-operator/tuned-pffvf","openshift-dns/node-resolver-m5mfn","openshift-multus/multus-additional-cni-plugins-7tgzl","openshift-network-diagnostics/network-check-target-fl4xc","openshift-ovn-kubernetes/ovnkube-node-fjw6m","kube-system/global-pull-secret-syncer-xq4v4","openshift-image-registry/node-ca-h7rmc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal","openshift-multus/multus-nz6p9","openshift-multus/network-metrics-daemon-rbkjh","openshift-network-operator/iptables-alerter-ks2bc"] Jul 28 17:17:24.147529 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.147511 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:24.147594 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.147575 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:24.148487 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.148470 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:24.148565 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.148534 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:24.149538 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.149521 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.150568 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.150551 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.151735 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.151718 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.152974 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.152612 2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 17:17:24.152974 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.152687 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.152974 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.152712 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 17:17:24.152974 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.152839 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.152974 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.152900 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6hp24\"" Jul 28 17:17:24.155318 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.154014 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.155318 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.154173 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sshjh\"" Jul 28 17:17:24.155318 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.154517 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.155318 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.154576 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.155318 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.154582 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.155318 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.154771 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hw2rj\"" Jul 28 17:17:24.156302 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.156271 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.157396 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.157378 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.157464 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.157440 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:24.158714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.158694 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.158796 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.158766 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 17:17:24.159108 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.159093 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t6sq4\"" Jul 28 17:17:24.159164 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.159112 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 17:17:24.159213 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.159164 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.159213 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.159184 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.159396 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.159379 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 17:17:24.159888 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.159857 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.160944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.160930 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.161330 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.161315 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 17:17:24.161394 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.161352 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cwk8s\"" Jul 28 17:17:24.161585 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.161570 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 17:17:24.162310 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.162295 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.163083 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163052 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.163083 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163055 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p5czz\"" Jul 28 17:17:24.163237 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163122 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 17:17:24.163237 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163203 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.163449 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163405 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.163653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163631 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85a1892b-1224-4b64-b5e4-27e9407042de-tmp\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.163739 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163665 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/014bd552-7857-4126-b50f-282e51e649bd-tmp-dir\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.163739 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163704 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 17:17:24.163739 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163706 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2288l\" (UniqueName: \"kubernetes.io/projected/014bd552-7857-4126-b50f-282e51e649bd-kube-api-access-2288l\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.163739 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163732 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163754 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jnc8\" (UniqueName: \"kubernetes.io/projected/85a1892b-1224-4b64-b5e4-27e9407042de-kube-api-access-8jnc8\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163707 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163778 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163800 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-systemd\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163822 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-system-cni-dir\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163861 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-os-release\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163923 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:24.163972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.163969 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-kubernetes\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164016 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-host\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164046 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cnibin\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164073 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq2hs\" (UniqueName: \"kubernetes.io/projected/37e9f3f8-63cd-4635-9a06-d59209a1d68f-kube-api-access-gq2hs\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164090 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzmch\" (UniqueName: \"kubernetes.io/projected/a7809890-59e0-45c7-9286-5f0fe38a4a64-kube-api-access-wzmch\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164104 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-registration-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164118 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-modprobe-d\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164155 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysctl-d\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164194 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-sys-fs\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164219 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysconfig\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164233 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysctl-conf\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164251 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164247 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-sys\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164262 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164278 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164292 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxjkj\" (UniqueName: \"kubernetes.io/projected/cf26a981-76c3-4b6d-b39d-08702d4037fc-kube-api-access-sxjkj\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164306 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-var-lib-kubelet\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164319 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/014bd552-7857-4126-b50f-282e51e649bd-hosts-file\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164334 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164364 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-device-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164383 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164399 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-lib-modules\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164414 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164437 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-socket-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164457 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-run\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.164609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164470 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85a1892b-1224-4b64-b5e4-27e9407042de-etc-tuned\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.165126 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164817 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.165126 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.164885 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8769f\"" Jul 28 17:17:24.166490 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166471 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gs794\"" Jul 28 17:17:24.166571 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166534 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 17:17:24.166624 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166568 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 17:17:24.166624 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166573 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 17:17:24.166624 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166578 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.166748 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166736 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 17:17:24.166811 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166796 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.166863 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.166797 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 17:17:24.167052 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.167038 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rchxq\"" Jul 28 17:17:24.170621 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.170606 2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:17:24.194799 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.194772 2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:17:24.217280 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.217258 2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kjvfk" Jul 28 17:17:24.233116 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.233095 2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kjvfk" Jul 28 17:17:24.253853 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.253810 2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 17:17:24.264819 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264794 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-kubelet\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.264944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264832 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-log-socket\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.264944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264858 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-cni-bin\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.264944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264896 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-kubelet\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.264944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264918 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-daemon-config\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264946 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysctl-conf\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264969 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-var-lib-kubelet\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.264991 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/014bd552-7857-4126-b50f-282e51e649bd-hosts-file\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265016 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-slash\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265038 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-etc-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265060 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-lib-modules\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265083 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265112 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-k8s-cni-cncf-io\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265137 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-etc-kubernetes\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265158 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-node-log\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265182 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85a1892b-1224-4b64-b5e4-27e9407042de-etc-tuned\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265206 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-system-cni-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265236 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jnc8\" (UniqueName: \"kubernetes.io/projected/85a1892b-1224-4b64-b5e4-27e9407042de-kube-api-access-8jnc8\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265261 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265284 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-os-release\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265322 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-cni-multus\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265347 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-hostroot\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265373 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-iptables-alerter-script\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265397 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-system-cni-dir\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265423 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265447 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-cnibin\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265468 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-netns\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265491 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-systemd\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265515 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cnibin\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265538 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-registration-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.265570 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265572 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-run-ovn-kubernetes\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265595 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cptmw\" (UniqueName: \"kubernetes.io/projected/01d66eca-4c69-40a8-989e-0783e7affc09-kube-api-access-cptmw\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265619 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-modprobe-d\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265642 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysctl-d\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265667 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgpgx\" (UniqueName: \"kubernetes.io/projected/42991c9d-3418-4060-affe-ba9b2e9a9bae-kube-api-access-jgpgx\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265691 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-run-netns\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265719 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-ovnkube-config\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265741 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01d66eca-4c69-40a8-989e-0783e7affc09-ovn-node-metrics-cert\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265764 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42991c9d-3418-4060-affe-ba9b2e9a9bae-cni-binary-copy\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265788 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-conf-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265813 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysconfig\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265838 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-sys\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265862 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265903 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265928 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxjkj\" (UniqueName: \"kubernetes.io/projected/cf26a981-76c3-4b6d-b39d-08702d4037fc-kube-api-access-sxjkj\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265952 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-cni-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.265977 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.266427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266018 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-device-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266042 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266067 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266094 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e32234d0-962e-463c-be7f-592397942fb4-host\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266117 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-cni-netd\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266142 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266170 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-socket-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266193 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e32234d0-962e-463c-be7f-592397942fb4-serviceca\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266217 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-var-lib-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266251 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266280 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-run\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266303 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85a1892b-1224-4b64-b5e4-27e9407042de-tmp\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266327 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/014bd552-7857-4126-b50f-282e51e649bd-tmp-dir\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266360 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2288l\" (UniqueName: \"kubernetes.io/projected/014bd552-7857-4126-b50f-282e51e649bd-kube-api-access-2288l\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266385 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266410 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-cni-bin\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266437 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-multus-certs\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.267273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266461 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fda79549-f59d-4702-b9a4-09416b782d4c-dbus\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266485 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-host-slash\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266509 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg8tc\" (UniqueName: \"kubernetes.io/projected/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-kube-api-access-lg8tc\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266534 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-systemd-units\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266564 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-env-overrides\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266588 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-ovnkube-script-lib\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266612 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-systemd\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266637 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-os-release\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266663 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6cb9de26-57d7-4d29-8b2d-0c953175f120-konnectivity-ca\") pod \"konnectivity-agent-dv5bc\" (UID: \"6cb9de26-57d7-4d29-8b2d-0c953175f120\") " pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266694 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-kubernetes\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266717 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-host\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266752 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq2hs\" (UniqueName: \"kubernetes.io/projected/37e9f3f8-63cd-4635-9a06-d59209a1d68f-kube-api-access-gq2hs\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266778 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzmch\" (UniqueName: \"kubernetes.io/projected/a7809890-59e0-45c7-9286-5f0fe38a4a64-kube-api-access-wzmch\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266806 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fda79549-f59d-4702-b9a4-09416b782d4c-kubelet-config\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266830 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6cb9de26-57d7-4d29-8b2d-0c953175f120-agent-certs\") pod \"konnectivity-agent-dv5bc\" (UID: \"6cb9de26-57d7-4d29-8b2d-0c953175f120\") " pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266861 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-ovn\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266905 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-sys-fs\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.268078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266933 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-socket-dir-parent\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.266959 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkcmb\" (UniqueName: \"kubernetes.io/projected/e32234d0-962e-463c-be7f-592397942fb4-kube-api-access-qkcmb\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.267159 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysctl-conf\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.267220 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-var-lib-kubelet\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.267262 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/014bd552-7857-4126-b50f-282e51e649bd-hosts-file\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.267383 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-lib-modules\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.267953 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268282 2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268552 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268671 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysctl-d\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268699 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268752 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-system-cni-dir\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268794 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-sysconfig\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268809 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-sys\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.268824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268819 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.268998 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269021 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/014bd552-7857-4126-b50f-282e51e649bd-tmp-dir\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269038 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-cnibin\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269082 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-socket-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269090 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-registration-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269124 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-device-dir\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269173 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-modprobe-d\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269183 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-run\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.269217 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:24.269481 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.269275 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs podName:a7809890-59e0-45c7-9286-5f0fe38a4a64 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:24.769255318 +0000 UTC m=+2.987296776 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs") pod "network-metrics-daemon-rbkjh" (UID: "a7809890-59e0-45c7-9286-5f0fe38a4a64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:24.269968 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269561 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-host\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.269968 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269863 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/37e9f3f8-63cd-4635-9a06-d59209a1d68f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.269968 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269949 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-systemd\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.269968 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269956 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cf26a981-76c3-4b6d-b39d-08702d4037fc-sys-fs\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.270152 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269992 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/37e9f3f8-63cd-4635-9a06-d59209a1d68f-os-release\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.270152 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.269999 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/85a1892b-1224-4b64-b5e4-27e9407042de-etc-kubernetes\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.273179 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.273158 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/85a1892b-1224-4b64-b5e4-27e9407042de-tmp\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.273270 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.273227 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/85a1892b-1224-4b64-b5e4-27e9407042de-etc-tuned\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.283196 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.283175 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:24.283317 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.283200 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:24.283317 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.283218 2554 projected.go:196] Error preparing data for projected volume kube-api-access-lttmp for pod openshift-network-diagnostics/network-check-target-fl4xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:24.283317 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.283269 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp podName:8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:24.783253732 +0000 UTC m=+3.001295177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lttmp" (UniqueName: "kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp") pod "network-check-target-fl4xc" (UID: "8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:24.285519 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.285498 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzmch\" (UniqueName: \"kubernetes.io/projected/a7809890-59e0-45c7-9286-5f0fe38a4a64-kube-api-access-wzmch\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:24.287033 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.287011 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq2hs\" (UniqueName: \"kubernetes.io/projected/37e9f3f8-63cd-4635-9a06-d59209a1d68f-kube-api-access-gq2hs\") pod \"multus-additional-cni-plugins-7tgzl\" (UID: \"37e9f3f8-63cd-4635-9a06-d59209a1d68f\") " pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.287033 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.287025 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2288l\" (UniqueName: \"kubernetes.io/projected/014bd552-7857-4126-b50f-282e51e649bd-kube-api-access-2288l\") pod \"node-resolver-m5mfn\" (UID: \"014bd552-7857-4126-b50f-282e51e649bd\") " pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.288589 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.288566 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxjkj\" (UniqueName: \"kubernetes.io/projected/cf26a981-76c3-4b6d-b39d-08702d4037fc-kube-api-access-sxjkj\") pod \"aws-ebs-csi-driver-node-x6vtt\" (UID: \"cf26a981-76c3-4b6d-b39d-08702d4037fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.288740 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.288649 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jnc8\" (UniqueName: \"kubernetes.io/projected/85a1892b-1224-4b64-b5e4-27e9407042de-kube-api-access-8jnc8\") pod \"tuned-pffvf\" (UID: \"85a1892b-1224-4b64-b5e4-27e9407042de\") " pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.337166 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.337103 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" event={"ID":"2901d1a7c56239f90dca1ec3946cd97a","Type":"ContainerStarted","Data":"23915e2ccfaef85c31645f939042c68001cb7a05b7a244f883fc72ee66631cfe"} Jul 28 17:17:24.338645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.338621 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" event={"ID":"fcb67e56a80fff0dbe2625d782081ff2","Type":"ContainerStarted","Data":"c265cff910b47ef49a7faa460c207a4071bbfab68f3019358442745d1544f80a"} Jul 28 17:17:24.367714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367662 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fda79549-f59d-4702-b9a4-09416b782d4c-kubelet-config\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.367714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367697 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6cb9de26-57d7-4d29-8b2d-0c953175f120-agent-certs\") pod \"konnectivity-agent-dv5bc\" (UID: \"6cb9de26-57d7-4d29-8b2d-0c953175f120\") " pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.367714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367726 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-ovn\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367753 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-socket-dir-parent\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367777 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkcmb\" (UniqueName: \"kubernetes.io/projected/e32234d0-962e-463c-be7f-592397942fb4-kube-api-access-qkcmb\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367801 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-kubelet\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367806 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fda79549-f59d-4702-b9a4-09416b782d4c-kubelet-config\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367825 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-log-socket\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367848 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-cni-bin\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367867 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-socket-dir-parent\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367885 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-kubelet\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367909 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-daemon-config\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367933 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-slash\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367954 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-etc-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367977 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-k8s-cni-cncf-io\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.367999 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-etc-kubernetes\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368022 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-node-log\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368044 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-system-cni-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368069 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-os-release\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368090 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-cni-multus\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368113 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-hostroot\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368138 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-iptables-alerter-script\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368175 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-cnibin\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368196 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-netns\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368221 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-systemd\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368248 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-run-ovn-kubernetes\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368271 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cptmw\" (UniqueName: \"kubernetes.io/projected/01d66eca-4c69-40a8-989e-0783e7affc09-kube-api-access-cptmw\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368318 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-etc-kubernetes\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368351 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgpgx\" (UniqueName: \"kubernetes.io/projected/42991c9d-3418-4060-affe-ba9b2e9a9bae-kube-api-access-jgpgx\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368374 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-run-netns\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368398 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-ovnkube-config\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368422 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01d66eca-4c69-40a8-989e-0783e7affc09-ovn-node-metrics-cert\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368446 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42991c9d-3418-4060-affe-ba9b2e9a9bae-cni-binary-copy\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368468 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-conf-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368510 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-cni-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368530 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-system-cni-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.368814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368539 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368562 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e32234d0-962e-463c-be7f-592397942fb4-host\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368587 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-cni-netd\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368615 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368630 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-cni-multus\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368641 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e32234d0-962e-463c-be7f-592397942fb4-serviceca\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368674 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-hostroot\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368702 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-var-lib-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368727 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368760 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-cni-bin\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368785 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-multus-certs\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368810 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fda79549-f59d-4702-b9a4-09416b782d4c-dbus\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368833 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-host-slash\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368859 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg8tc\" (UniqueName: \"kubernetes.io/projected/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-kube-api-access-lg8tc\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368903 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-systemd-units\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368929 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-env-overrides\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368952 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-ovnkube-script-lib\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.369662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368984 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6cb9de26-57d7-4d29-8b2d-0c953175f120-konnectivity-ca\") pod \"konnectivity-agent-dv5bc\" (UID: \"6cb9de26-57d7-4d29-8b2d-0c953175f120\") " pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369040 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e32234d0-962e-463c-be7f-592397942fb4-serviceca\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369526 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6cb9de26-57d7-4d29-8b2d-0c953175f120-konnectivity-ca\") pod \"konnectivity-agent-dv5bc\" (UID: \"6cb9de26-57d7-4d29-8b2d-0c953175f120\") " pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369583 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-iptables-alerter-script\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369584 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-kubelet\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369615 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-log-socket\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369653 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-cnibin\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369690 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-kubelet\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369725 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-k8s-cni-cncf-io\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369726 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-etc-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369758 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-netns\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369799 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-systemd\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369837 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-run-ovn-kubernetes\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369892 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-var-lib-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369927 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-openvswitch\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369963 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-var-lib-cni-bin\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369654 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-cni-bin\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.369692 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-slash\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.370506 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.370095 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:24.371322 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.370142 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret podName:fda79549-f59d-4702-b9a4-09416b782d4c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:24.870126444 +0000 UTC m=+3.088167902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret") pod "global-pull-secret-syncer-xq4v4" (UID: "fda79549-f59d-4702-b9a4-09416b782d4c") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:24.371322 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.371127 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-ovnkube-config\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.376735 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.376694 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/42991c9d-3418-4060-affe-ba9b2e9a9bae-cni-binary-copy\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.376832 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.376772 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-conf-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.376943 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.376838 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-node-log\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.376943 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.376901 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-run-netns\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.377063 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.377046 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fda79549-f59d-4702-b9a4-09416b782d4c-dbus\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.377125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.377110 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-host-slash\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.377475 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.377416 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-cni-dir\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.377545 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.377479 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-systemd-units\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.378754 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.378724 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-env-overrides\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.379258 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.379237 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/42991c9d-3418-4060-affe-ba9b2e9a9bae-multus-daemon-config\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.380507 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.380484 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e32234d0-962e-463c-be7f-592397942fb4-host\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.380585 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.380558 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-cni-netd\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.380645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.368589 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-os-release\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.380733 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.380702 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/42991c9d-3418-4060-affe-ba9b2e9a9bae-host-run-multus-certs\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.380866 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.380845 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.381313 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.381290 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01d66eca-4c69-40a8-989e-0783e7affc09-run-ovn\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.382036 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.382011 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01d66eca-4c69-40a8-989e-0783e7affc09-ovnkube-script-lib\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.382456 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.382435 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6cb9de26-57d7-4d29-8b2d-0c953175f120-agent-certs\") pod \"konnectivity-agent-dv5bc\" (UID: \"6cb9de26-57d7-4d29-8b2d-0c953175f120\") " pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.383820 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.383797 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01d66eca-4c69-40a8-989e-0783e7affc09-ovn-node-metrics-cert\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.385436 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.385412 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgpgx\" (UniqueName: \"kubernetes.io/projected/42991c9d-3418-4060-affe-ba9b2e9a9bae-kube-api-access-jgpgx\") pod \"multus-nz6p9\" (UID: \"42991c9d-3418-4060-affe-ba9b2e9a9bae\") " pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.387031 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.386981 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cptmw\" (UniqueName: \"kubernetes.io/projected/01d66eca-4c69-40a8-989e-0783e7affc09-kube-api-access-cptmw\") pod \"ovnkube-node-fjw6m\" (UID: \"01d66eca-4c69-40a8-989e-0783e7affc09\") " pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.387431 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.387398 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg8tc\" (UniqueName: \"kubernetes.io/projected/fe3167de-ca14-4cb3-a1cb-f971c33e04d1-kube-api-access-lg8tc\") pod \"iptables-alerter-ks2bc\" (UID: \"fe3167de-ca14-4cb3-a1cb-f971c33e04d1\") " pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.394312 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.394270 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkcmb\" (UniqueName: \"kubernetes.io/projected/e32234d0-962e-463c-be7f-592397942fb4-kube-api-access-qkcmb\") pod \"node-ca-h7rmc\" (UID: \"e32234d0-962e-463c-be7f-592397942fb4\") " pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.460643 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.460616 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" Jul 28 17:17:24.466240 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.466220 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pffvf" Jul 28 17:17:24.473329 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.473306 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-m5mfn" Jul 28 17:17:24.477004 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.476948 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" Jul 28 17:17:24.478605 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:24.478578 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf26a981_76c3_4b6d_b39d_08702d4037fc.slice/crio-ed2012bd9531261ef654871689b27acb9a4c69cbcaf640b07973771f5d243572 WatchSource:0}: Error finding container ed2012bd9531261ef654871689b27acb9a4c69cbcaf640b07973771f5d243572: Status 404 returned error can't find the container with id ed2012bd9531261ef654871689b27acb9a4c69cbcaf640b07973771f5d243572 Jul 28 17:17:24.482599 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.482578 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:24.485729 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:24.485704 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod014bd552_7857_4126_b50f_282e51e649bd.slice/crio-37f7036287b1072bfb1ab93b909bfbfc06b905cd9f873d17666c11f0b248c224 WatchSource:0}: Error finding container 37f7036287b1072bfb1ab93b909bfbfc06b905cd9f873d17666c11f0b248c224: Status 404 returned error can't find the container with id 37f7036287b1072bfb1ab93b909bfbfc06b905cd9f873d17666c11f0b248c224 Jul 28 17:17:24.489588 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.489568 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ks2bc" Jul 28 17:17:24.499555 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.499537 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h7rmc" Jul 28 17:17:24.503512 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:24.503457 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37e9f3f8_63cd_4635_9a06_d59209a1d68f.slice/crio-8e23a17c24638d6c5f13ce0793a95457baec02bb1749cbc8965b603ecb315206 WatchSource:0}: Error finding container 8e23a17c24638d6c5f13ce0793a95457baec02bb1749cbc8965b603ecb315206: Status 404 returned error can't find the container with id 8e23a17c24638d6c5f13ce0793a95457baec02bb1749cbc8965b603ecb315206 Jul 28 17:17:24.504824 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:24.504803 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3167de_ca14_4cb3_a1cb_f971c33e04d1.slice/crio-1b3d5d254ca9360554a62f7c3909803505bcb363a17a3eb47c924ccb07a713ad WatchSource:0}: Error finding container 1b3d5d254ca9360554a62f7c3909803505bcb363a17a3eb47c924ccb07a713ad: Status 404 returned error can't find the container with id 1b3d5d254ca9360554a62f7c3909803505bcb363a17a3eb47c924ccb07a713ad Jul 28 17:17:24.505943 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.505927 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:24.506310 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:24.506290 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb9de26_57d7_4d29_8b2d_0c953175f120.slice/crio-6fd07a357412be3a2505268e98af5791d9b0587ef465c6b6cf26d5f230736071 WatchSource:0}: Error finding container 6fd07a357412be3a2505268e98af5791d9b0587ef465c6b6cf26d5f230736071: Status 404 returned error can't find the container with id 6fd07a357412be3a2505268e98af5791d9b0587ef465c6b6cf26d5f230736071 Jul 28 17:17:24.512766 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.512748 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nz6p9" Jul 28 17:17:24.525223 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:24.525190 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42991c9d_3418_4060_affe_ba9b2e9a9bae.slice/crio-87be8904b5b36f0d0790b0a4b7dd251bcdd2f3851ac5c965da00eb2ce54a0d57 WatchSource:0}: Error finding container 87be8904b5b36f0d0790b0a4b7dd251bcdd2f3851ac5c965da00eb2ce54a0d57: Status 404 returned error can't find the container with id 87be8904b5b36f0d0790b0a4b7dd251bcdd2f3851ac5c965da00eb2ce54a0d57 Jul 28 17:17:24.772479 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.772442 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:24.772643 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.772587 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:24.772703 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.772652 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs podName:a7809890-59e0-45c7-9286-5f0fe38a4a64 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:25.772637305 +0000 UTC m=+3.990678746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs") pod "network-metrics-daemon-rbkjh" (UID: "a7809890-59e0-45c7-9286-5f0fe38a4a64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:24.873284 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.873256 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:24.873455 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:24.873329 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:24.873455 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.873411 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:24.873455 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.873438 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:24.873455 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.873452 2554 projected.go:196] Error preparing data for projected volume kube-api-access-lttmp for pod openshift-network-diagnostics/network-check-target-fl4xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:24.873455 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.873453 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:24.873712 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.873515 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp podName:8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:25.873495185 +0000 UTC m=+4.091536627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lttmp" (UniqueName: "kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp") pod "network-check-target-fl4xc" (UID: "8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:24.873712 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:24.873536 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret podName:fda79549-f59d-4702-b9a4-09416b782d4c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:25.873526999 +0000 UTC m=+4.091568459 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret") pod "global-pull-secret-syncer-xq4v4" (UID: "fda79549-f59d-4702-b9a4-09416b782d4c") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:25.235011 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.234924 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:12:24 +0000 UTC" deadline="2028-04-03 08:23:59.799400982 +0000 UTC" Jul 28 17:17:25.235011 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.234966 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14751h6m34.564439596s" Jul 28 17:17:25.332295 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.332268 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:25.332442 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.332397 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:25.332763 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.332745 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:25.332862 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.332841 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:25.343813 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.343765 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dv5bc" event={"ID":"6cb9de26-57d7-4d29-8b2d-0c953175f120","Type":"ContainerStarted","Data":"6fd07a357412be3a2505268e98af5791d9b0587ef465c6b6cf26d5f230736071"} Jul 28 17:17:25.346815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.346790 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m5mfn" event={"ID":"014bd552-7857-4126-b50f-282e51e649bd","Type":"ContainerStarted","Data":"37f7036287b1072bfb1ab93b909bfbfc06b905cd9f873d17666c11f0b248c224"} Jul 28 17:17:25.349590 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.349566 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pffvf" event={"ID":"85a1892b-1224-4b64-b5e4-27e9407042de","Type":"ContainerStarted","Data":"62fd40831065acbc66409cbc2d2dfec3fd8f931dee0b71d58cfa0586af4f3e0b"} Jul 28 17:17:25.352122 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.352097 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" event={"ID":"cf26a981-76c3-4b6d-b39d-08702d4037fc","Type":"ContainerStarted","Data":"ed2012bd9531261ef654871689b27acb9a4c69cbcaf640b07973771f5d243572"} Jul 28 17:17:25.353996 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.353972 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ks2bc" event={"ID":"fe3167de-ca14-4cb3-a1cb-f971c33e04d1","Type":"ContainerStarted","Data":"1b3d5d254ca9360554a62f7c3909803505bcb363a17a3eb47c924ccb07a713ad"} Jul 28 17:17:25.355306 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.355283 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerStarted","Data":"8e23a17c24638d6c5f13ce0793a95457baec02bb1749cbc8965b603ecb315206"} Jul 28 17:17:25.356927 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.356904 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nz6p9" event={"ID":"42991c9d-3418-4060-affe-ba9b2e9a9bae","Type":"ContainerStarted","Data":"87be8904b5b36f0d0790b0a4b7dd251bcdd2f3851ac5c965da00eb2ce54a0d57"} Jul 28 17:17:25.358382 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.358357 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"2ff3ef13fb4047ce0454ef7238256b0d73c8addaa55bc2a1ced780f2c194dacf"} Jul 28 17:17:25.359615 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.359591 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h7rmc" event={"ID":"e32234d0-962e-463c-be7f-592397942fb4","Type":"ContainerStarted","Data":"f583dc242089c07899a89577c74d7ae7d02c268d22cc267d0090815fe843cb9b"} Jul 28 17:17:25.782155 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.782119 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:25.782386 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.782330 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:25.782480 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.782391 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs podName:a7809890-59e0-45c7-9286-5f0fe38a4a64 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:27.782372427 +0000 UTC m=+6.000413871 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs") pod "network-metrics-daemon-rbkjh" (UID: "a7809890-59e0-45c7-9286-5f0fe38a4a64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:25.883114 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.883067 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:25.883258 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:25.883147 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:25.883329 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.883300 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:25.883377 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.883361 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret podName:fda79549-f59d-4702-b9a4-09416b782d4c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:27.883341898 +0000 UTC m=+6.101383359 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret") pod "global-pull-secret-syncer-xq4v4" (UID: "fda79549-f59d-4702-b9a4-09416b782d4c") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:25.883463 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.883448 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:25.883525 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.883472 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:25.883525 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.883484 2554 projected.go:196] Error preparing data for projected volume kube-api-access-lttmp for pod openshift-network-diagnostics/network-check-target-fl4xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:25.883614 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:25.883533 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp podName:8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:27.883510569 +0000 UTC m=+6.101552027 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lttmp" (UniqueName: "kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp") pod "network-check-target-fl4xc" (UID: "8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:26.235687 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:26.235575 2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:12:24 +0000 UTC" deadline="2028-03-20 02:03:31.417947943 +0000 UTC" Jul 28 17:17:26.235687 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:26.235639 2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14408h46m5.182312957s" Jul 28 17:17:26.334533 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:26.334501 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:26.334683 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:26.334644 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:27.332536 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:27.332505 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:27.333055 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.332635 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:27.333128 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:27.333095 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:27.333420 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.333259 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:27.799098 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:27.799064 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:27.799268 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.799245 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:27.799339 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.799308 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs podName:a7809890-59e0-45c7-9286-5f0fe38a4a64 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:31.799289876 +0000 UTC m=+10.017331321 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs") pod "network-metrics-daemon-rbkjh" (UID: "a7809890-59e0-45c7-9286-5f0fe38a4a64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:27.900204 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:27.900170 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:27.900350 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:27.900239 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:27.900423 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.900382 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:27.900474 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.900441 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret podName:fda79549-f59d-4702-b9a4-09416b782d4c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:31.900422096 +0000 UTC m=+10.118463551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret") pod "global-pull-secret-syncer-xq4v4" (UID: "fda79549-f59d-4702-b9a4-09416b782d4c") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:27.900891 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.900850 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:27.900891 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.900888 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:27.901057 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.900903 2554 projected.go:196] Error preparing data for projected volume kube-api-access-lttmp for pod openshift-network-diagnostics/network-check-target-fl4xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:27.901057 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:27.900989 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp podName:8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:31.900974821 +0000 UTC m=+10.119016264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lttmp" (UniqueName: "kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp") pod "network-check-target-fl4xc" (UID: "8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:28.332332 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:28.332302 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:28.332509 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:28.332449 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:29.331842 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:29.331805 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:29.332298 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:29.331944 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:29.332298 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:29.332140 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:29.332298 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:29.332247 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:30.332615 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:30.332584 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:30.333042 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:30.332736 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:31.332234 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:31.331630 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:31.332234 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.331770 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:31.332234 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:31.332153 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:31.332565 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.332263 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:31.833125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:31.833090 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:31.833579 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.833266 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:31.833579 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.833336 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs podName:a7809890-59e0-45c7-9286-5f0fe38a4a64 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:39.833315105 +0000 UTC m=+18.051356547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs") pod "network-metrics-daemon-rbkjh" (UID: "a7809890-59e0-45c7-9286-5f0fe38a4a64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:31.934296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:31.934265 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:31.934443 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:31.934330 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:31.934443 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.934420 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:31.934562 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.934477 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret podName:fda79549-f59d-4702-b9a4-09416b782d4c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:39.934458641 +0000 UTC m=+18.152500106 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret") pod "global-pull-secret-syncer-xq4v4" (UID: "fda79549-f59d-4702-b9a4-09416b782d4c") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:31.934562 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.934423 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:31.934562 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.934512 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:31.934562 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.934525 2554 projected.go:196] Error preparing data for projected volume kube-api-access-lttmp for pod openshift-network-diagnostics/network-check-target-fl4xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:31.934758 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:31.934567 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp podName:8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:39.934554656 +0000 UTC m=+18.152596104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lttmp" (UniqueName: "kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp") pod "network-check-target-fl4xc" (UID: "8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:32.332159 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:32.332127 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:32.332336 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:32.332274 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:33.331866 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:33.331834 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:33.332280 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:33.331846 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:33.332280 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:33.331992 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:33.332280 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:33.332053 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:34.334287 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:34.334256 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:34.334649 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:34.334406 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:35.331971 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:35.331938 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:35.332156 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:35.332061 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:35.332156 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:35.332126 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:35.332275 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:35.332221 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:36.332063 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:36.331993 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:36.332516 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:36.332136 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:37.331844 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:37.331811 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:37.331991 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:37.331854 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:37.331991 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:37.331952 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:37.332089 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:37.332074 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:38.332386 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:38.332352 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:38.332819 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:38.332486 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:39.332323 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:39.332286 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:39.332527 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:39.332286 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:39.332527 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.332417 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:39.332527 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.332451 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:39.897528 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:39.897490 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:39.897681 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.897640 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:39.897746 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.897697 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs podName:a7809890-59e0-45c7-9286-5f0fe38a4a64 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:55.897683466 +0000 UTC m=+34.115724910 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs") pod "network-metrics-daemon-rbkjh" (UID: "a7809890-59e0-45c7-9286-5f0fe38a4a64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:39.998307 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:39.998274 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:39.998494 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:39.998351 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:39.998494 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.998458 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:39.998494 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.998468 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:39.998494 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.998487 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:39.998494 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.998498 2554 projected.go:196] Error preparing data for projected volume kube-api-access-lttmp for pod openshift-network-diagnostics/network-check-target-fl4xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:39.998742 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.998527 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret podName:fda79549-f59d-4702-b9a4-09416b782d4c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:55.998508276 +0000 UTC m=+34.216549734 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret") pod "global-pull-secret-syncer-xq4v4" (UID: "fda79549-f59d-4702-b9a4-09416b782d4c") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:39.998742 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:39.998546 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp podName:8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c nodeName:}" failed. No retries permitted until 2026-07-28 17:17:55.998537818 +0000 UTC m=+34.216579262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lttmp" (UniqueName: "kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp") pod "network-check-target-fl4xc" (UID: "8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:40.331882 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:40.331835 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:40.332039 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:40.331974 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:41.332354 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:41.332318 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:41.332755 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:41.332421 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:41.332755 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:41.332491 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:41.332755 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:41.332604 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:42.332707 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:42.332629 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:42.333207 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:42.332752 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:43.332629 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.332466 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:43.332735 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.332466 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:43.332735 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:43.332691 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:43.333088 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:43.332780 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:43.395762 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.395690 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nz6p9" event={"ID":"42991c9d-3418-4060-affe-ba9b2e9a9bae","Type":"ContainerStarted","Data":"adab9fae2dedebfd89f486cf8fc438cc13048247057c415142dd1b62bc1f262c"} Jul 28 17:17:43.397266 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.397244 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pffvf" event={"ID":"85a1892b-1224-4b64-b5e4-27e9407042de","Type":"ContainerStarted","Data":"f965a80cf671549f3d7ca921c6137d50a138b8f57e3573460947972b609a29b5"} Jul 28 17:17:43.398502 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.398482 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" event={"ID":"fcb67e56a80fff0dbe2625d782081ff2","Type":"ContainerStarted","Data":"e14d2b4de4012c6bdbd7bec0ff7e9d65cb4e6fd567fae2708ed02a0e06775e81"} Jul 28 17:17:43.416820 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.416785 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nz6p9" podStartSLOduration=2.9588266450000003 podStartE2EDuration="21.416773759s" totalImagesPullingTime="18.457947114s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:43.416404154 +0000 UTC m=+21.634445614" watchObservedRunningTime="2026-07-28 17:17:43.416773759 +0000 UTC m=+21.634815224" Jul 28 17:17:43.438998 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.438926 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pffvf" podStartSLOduration=3.02271027 podStartE2EDuration="21.438913695s" totalImagesPullingTime="18.416203425s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:43.438725311 +0000 UTC m=+21.656766775" watchObservedRunningTime="2026-07-28 17:17:43.438913695 +0000 UTC m=+21.656955159" Jul 28 17:17:43.454589 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:43.454545 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-206.ec2.internal" podStartSLOduration=20.454528982 podStartE2EDuration="20.454528982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:43.454482898 +0000 UTC m=+21.672524362" watchObservedRunningTime="2026-07-28 17:17:43.454528982 +0000 UTC m=+21.672570449" Jul 28 17:17:44.332340 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.332070 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:44.332458 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:44.332412 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:44.401630 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.401606 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" event={"ID":"cf26a981-76c3-4b6d-b39d-08702d4037fc","Type":"ContainerStarted","Data":"77d674634f3d6e2aadb01f0394ed261a4370f5d529a7322696b3bc0e2c9c9b57"} Jul 28 17:17:44.403003 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.402974 2554 generic.go:356] "Generic (PLEG): container finished" podID="2901d1a7c56239f90dca1ec3946cd97a" containerID="50d21424202c755a5673486198aebe0eba2628d1a7e9780d93ab7bcb651c1693" exitCode=0 Jul 28 17:17:44.403081 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.403051 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" event={"ID":"2901d1a7c56239f90dca1ec3946cd97a","Type":"ContainerDied","Data":"50d21424202c755a5673486198aebe0eba2628d1a7e9780d93ab7bcb651c1693"} Jul 28 17:17:44.404428 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.404409 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ks2bc" event={"ID":"fe3167de-ca14-4cb3-a1cb-f971c33e04d1","Type":"ContainerStarted","Data":"80f646a6768859e5e9033aa34b4d494cc3b4997e049fdab59f0a1e328a275671"} Jul 28 17:17:44.405909 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.405890 2554 generic.go:356] "Generic (PLEG): container finished" podID="37e9f3f8-63cd-4635-9a06-d59209a1d68f" containerID="cbf93f662cdf0454820c36a42e09222656220475dc5d5228cfca6b0cb657b81b" exitCode=0 Jul 28 17:17:44.406008 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.405959 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerDied","Data":"cbf93f662cdf0454820c36a42e09222656220475dc5d5228cfca6b0cb657b81b"} Jul 28 17:17:44.408861 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.408841 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"70fafe1324be20731c46c867c942c91304ae505e1473ce31698f3ba68d430659"} Jul 28 17:17:44.408983 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.408912 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"c59626ec24dc5a3f1bcf7fc1e05bc224f369fcf4a71403af2399665887f9dad8"} Jul 28 17:17:44.408983 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.408954 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"439143b560c0923e2264da4f6f7c64019b10511fa2dc6da0f19e681f38df5df2"} Jul 28 17:17:44.409086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.409002 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"f986b7973ad4bd10ffea0a67353fa229631f80916ceb53e75f6d53e2dd416baa"} Jul 28 17:17:44.409086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.409038 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"7ebd6c81b66072b6523168ae1ff4a5936da612d9aa13d47a3ad9869ac014cf39"} Jul 28 17:17:44.409086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.409074 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"a307491e0847412dbf9f1901013009e7af565bf932fb92a28c2bc079bb24c20a"} Jul 28 17:17:44.410117 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.410095 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h7rmc" event={"ID":"e32234d0-962e-463c-be7f-592397942fb4","Type":"ContainerStarted","Data":"130d8678a9262ea2fe4af583b217b7630aa78def324e68984f4c53e8662b012e"} Jul 28 17:17:44.411335 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.411312 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dv5bc" event={"ID":"6cb9de26-57d7-4d29-8b2d-0c953175f120","Type":"ContainerStarted","Data":"8c6b28ddd49fc051ef1ccd4410ccfcf5161fc05587519563be92f373385d05c2"} Jul 28 17:17:44.412634 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.412615 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-m5mfn" event={"ID":"014bd552-7857-4126-b50f-282e51e649bd","Type":"ContainerStarted","Data":"331b2165c291d560a2e5f2af59de036ff6e2232516368798065afbce0a8c751a"} Jul 28 17:17:44.458126 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.458091 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dv5bc" podStartSLOduration=12.771612654 podStartE2EDuration="22.458079629s" totalImagesPullingTime="9.686466975s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:44.457682887 +0000 UTC m=+22.675724349" watchObservedRunningTime="2026-07-28 17:17:44.458079629 +0000 UTC m=+22.676121092" Jul 28 17:17:44.473160 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.473128 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ks2bc" podStartSLOduration=12.785815403 podStartE2EDuration="22.473116598s" totalImagesPullingTime="9.687301195s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:44.473032774 +0000 UTC m=+22.691074237" watchObservedRunningTime="2026-07-28 17:17:44.473116598 +0000 UTC m=+22.691158060" Jul 28 17:17:44.488314 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.488280 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-m5mfn" podStartSLOduration=4.077533319 podStartE2EDuration="22.488269614s" totalImagesPullingTime="18.410736295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:44.488091527 +0000 UTC m=+22.706132989" watchObservedRunningTime="2026-07-28 17:17:44.488269614 +0000 UTC m=+22.706311076" Jul 28 17:17:44.497826 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.497806 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:44.599757 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.599697 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:44.682202 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.682181 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:44.745483 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:44.745455 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:45.332298 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:45.332261 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:45.332298 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:45.332280 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:45.332601 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:45.332390 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:45.332601 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:45.332504 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:45.416328 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:45.416274 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" event={"ID":"2901d1a7c56239f90dca1ec3946cd97a","Type":"ContainerStarted","Data":"baccbeb2558a74d1483cd5a87178db68b0bc0549304a6b20d9485fc84552948a"} Jul 28 17:17:45.434451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:45.434380 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h7rmc" podStartSLOduration=9.025447582 podStartE2EDuration="23.43436105s" totalImagesPullingTime="14.408913468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:44.505832379 +0000 UTC m=+22.723873843" watchObservedRunningTime="2026-07-28 17:17:45.43436105 +0000 UTC m=+23.652402514" Jul 28 17:17:45.434574 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:45.434479 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-206.ec2.internal" podStartSLOduration=22.434471971 podStartE2EDuration="22.434471971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:45.4340635 +0000 UTC m=+23.652104963" watchObservedRunningTime="2026-07-28 17:17:45.434471971 +0000 UTC m=+23.652513436" Jul 28 17:17:45.531010 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:45.530973 2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 17:17:46.230730 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:46.230640 2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:17:45.530993234Z","UUID":"e51e8e37-1d4d-4b1a-bf7c-50f1e239a9fe","Handler":null,"Name":"","Endpoint":""} Jul 28 17:17:46.232101 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:46.232073 2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 17:17:46.232226 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:46.232118 2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 17:17:46.332374 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:46.332327 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:46.332525 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:46.332485 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:46.421221 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:46.421174 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" event={"ID":"cf26a981-76c3-4b6d-b39d-08702d4037fc","Type":"ContainerStarted","Data":"34537af6ce34673b06d13c4749c81c0f8a97bcd6755cfd28657029bf5c4e6844"} Jul 28 17:17:47.332145 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:47.332106 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:47.332323 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:47.332237 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:47.332323 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:47.332291 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:47.332436 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:47.332397 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:47.426098 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:47.426062 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" event={"ID":"cf26a981-76c3-4b6d-b39d-08702d4037fc","Type":"ContainerStarted","Data":"1eda621c3cbccb3bb040fb2e707c78a531142ee664f38fb7c02966fbba457e82"} Jul 28 17:17:47.429652 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:47.429625 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"0a72883ee601d59485571c9eb1a2b5e8d0467b092b73a81b5c6e9f5db88040e5"} Jul 28 17:17:47.777435 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:47.777406 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:47.777996 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:47.777977 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:47.794831 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:47.794785 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x6vtt" podStartSLOduration=4.66988245 podStartE2EDuration="25.794766636s" totalImagesPullingTime="21.124884186s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:47.446956996 +0000 UTC m=+25.664998459" watchObservedRunningTime="2026-07-28 17:17:47.794766636 +0000 UTC m=+26.012808100" Jul 28 17:17:48.332591 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:48.332550 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:48.332766 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:48.332704 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:48.432422 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:48.432394 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dv5bc" Jul 28 17:17:48.523919 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:48.523890 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:49.332319 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.332154 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:49.332469 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.332156 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:49.332469 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:49.332388 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:49.332538 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:49.332476 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:49.437384 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.437350 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" event={"ID":"01d66eca-4c69-40a8-989e-0783e7affc09","Type":"ContainerStarted","Data":"7921080b32e87ee14dad564a6b74fd213bfb5c647cef112063d1e5b3812e72c2"} Jul 28 17:17:49.438347 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.437572 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:49.439112 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.439092 2554 generic.go:356] "Generic (PLEG): container finished" podID="37e9f3f8-63cd-4635-9a06-d59209a1d68f" containerID="ad257d600b4908a249e48d202465ee9bf30c142638143eeace0ec29bd662fcb1" exitCode=0 Jul 28 17:17:49.439185 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.439146 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerDied","Data":"ad257d600b4908a249e48d202465ee9bf30c142638143eeace0ec29bd662fcb1"} Jul 28 17:17:49.452367 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.452349 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:49.473955 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:49.472091 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" podStartSLOduration=8.48072998 podStartE2EDuration="27.472074625s" totalImagesPullingTime="18.991344645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:49.471713951 +0000 UTC m=+27.689755423" watchObservedRunningTime="2026-07-28 17:17:49.472074625 +0000 UTC m=+27.690116089" Jul 28 17:17:50.332196 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:50.332025 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:50.332370 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:50.332313 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:50.442081 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:50.442053 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:50.442349 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:50.442127 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:50.456032 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:50.456011 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:17:51.155000 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.154972 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xq4v4"] Jul 28 17:17:51.155155 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.155081 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:51.155215 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:51.155182 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:51.158185 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.158156 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fl4xc"] Jul 28 17:17:51.161476 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.158686 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:51.161476 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:51.158825 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:51.161476 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.159780 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rbkjh"] Jul 28 17:17:51.161476 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.159947 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:51.161476 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:51.160076 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:51.445465 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.445387 2554 generic.go:356] "Generic (PLEG): container finished" podID="37e9f3f8-63cd-4635-9a06-d59209a1d68f" containerID="26893f5def3fd5fa35410172812895c2c5eeb9200ab0394ba18afc9638b69393" exitCode=0 Jul 28 17:17:51.445465 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.445451 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerDied","Data":"26893f5def3fd5fa35410172812895c2c5eeb9200ab0394ba18afc9638b69393"} Jul 28 17:17:51.538124 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.538096 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:51.634595 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:51.634567 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:52.333125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:52.333091 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:52.333279 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:52.333181 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:52.333324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:52.333284 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:52.333415 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:52.333395 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:52.449709 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:52.449643 2554 generic.go:356] "Generic (PLEG): container finished" podID="37e9f3f8-63cd-4635-9a06-d59209a1d68f" containerID="11b2d9a23bc026016127a347f01daeee03819dad5965628487839e6c3f376ffd" exitCode=0 Jul 28 17:17:52.450060 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:52.449711 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerDied","Data":"11b2d9a23bc026016127a347f01daeee03819dad5965628487839e6c3f376ffd"} Jul 28 17:17:53.332142 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:53.332058 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:53.332294 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:53.332179 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:54.332644 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:54.332607 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:54.333131 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:54.332711 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:54.333131 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:54.332718 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:54.333131 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:54.332793 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:54.754365 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:54.754332 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:54.843491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:54.843328 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:55.331657 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:55.331579 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:55.331819 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:55.331717 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rbkjh" podUID="a7809890-59e0-45c7-9286-5f0fe38a4a64" Jul 28 17:17:55.937099 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:55.937066 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:55.937518 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:55.937205 2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:55.937518 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:55.937261 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs podName:a7809890-59e0-45c7-9286-5f0fe38a4a64 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:27.937243305 +0000 UTC m=+66.155284745 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs") pod "network-metrics-daemon-rbkjh" (UID: "a7809890-59e0-45c7-9286-5f0fe38a4a64") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:17:56.037790 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:56.037760 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:56.037961 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:56.037804 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:56.037961 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.037928 2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:56.037961 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.037940 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:17:56.037961 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.037955 2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:17:56.038174 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.037967 2554 projected.go:196] Error preparing data for projected volume kube-api-access-lttmp for pod openshift-network-diagnostics/network-check-target-fl4xc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:56.038174 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.037987 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret podName:fda79549-f59d-4702-b9a4-09416b782d4c nodeName:}" failed. No retries permitted until 2026-07-28 17:18:28.037970581 +0000 UTC m=+66.256012039 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret") pod "global-pull-secret-syncer-xq4v4" (UID: "fda79549-f59d-4702-b9a4-09416b782d4c") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:17:56.038174 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.038003 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp podName:8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c nodeName:}" failed. No retries permitted until 2026-07-28 17:18:28.037993351 +0000 UTC m=+66.256034792 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lttmp" (UniqueName: "kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp") pod "network-check-target-fl4xc" (UID: "8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:17:56.331890 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:56.331846 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:56.331890 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:56.331867 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:56.332107 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.331959 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fl4xc" podUID="8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c" Jul 28 17:17:56.332107 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:56.332021 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xq4v4" podUID="fda79549-f59d-4702-b9a4-09416b782d4c" Jul 28 17:17:56.589480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:56.589437 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m5mfn_014bd552-7857-4126-b50f-282e51e649bd/dns-node-resolver/0.log" Jul 28 17:17:57.110471 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.110420 2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-206.ec2.internal" event="NodeReady" Jul 28 17:17:57.110824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.110505 2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 17:17:57.165663 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.165634 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dzxzw"] Jul 28 17:17:57.186534 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.186511 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rflm7"] Jul 28 17:17:57.186649 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.186631 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.189264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.189243 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 17:17:57.189410 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.189396 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 17:17:57.189624 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.189612 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qjb4d\"" Jul 28 17:17:57.190219 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.190205 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h7rmc_e32234d0-962e-463c-be7f-592397942fb4/node-ca/0.log" Jul 28 17:17:57.212671 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.212654 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dzxzw"] Jul 28 17:17:57.212759 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.212684 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rflm7"] Jul 28 17:17:57.212797 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.212759 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.215436 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.215419 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 17:17:57.215541 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.215524 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 17:17:57.215590 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.215538 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 17:17:57.215634 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.215614 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mc5b9\"" Jul 28 17:17:57.215670 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.215655 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 17:17:57.245250 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245229 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46hkk\" (UniqueName: \"kubernetes.io/projected/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-api-access-46hkk\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.245353 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245264 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7eec1800-06ec-47c6-bad2-dab46839c8b2-config-volume\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.245353 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245280 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.245353 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245296 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.245353 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245311 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.245524 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245353 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eec1800-06ec-47c6-bad2-dab46839c8b2-tmp-dir\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.245524 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245381 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cjv9\" (UniqueName: \"kubernetes.io/projected/7eec1800-06ec-47c6-bad2-dab46839c8b2-kube-api-access-9cjv9\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.245524 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245397 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0989e1e-1e45-401f-b97b-c8ec8500e786-data-volume\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.245524 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.245451 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c0989e1e-1e45-401f-b97b-c8ec8500e786-crio-socket\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.288479 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.288455 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4z9q8"] Jul 28 17:17:57.304569 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.304552 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4z9q8"] Jul 28 17:17:57.304674 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.304650 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.307569 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.307543 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 17:17:57.307901 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.307866 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 17:17:57.307974 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.307934 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 17:17:57.308032 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.308021 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wqh9j\"" Jul 28 17:17:57.331944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.331928 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:17:57.335027 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.335012 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:17:57.335099 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.335067 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwzr\"" Jul 28 17:17:57.346215 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346198 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46hkk\" (UniqueName: \"kubernetes.io/projected/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-api-access-46hkk\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.346296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346240 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7eec1800-06ec-47c6-bad2-dab46839c8b2-config-volume\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.346296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346256 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.346382 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.346325 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:57.346382 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.346366 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls podName:7eec1800-06ec-47c6-bad2-dab46839c8b2 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:57.846350209 +0000 UTC m=+36.064391649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls") pod "dns-default-dzxzw" (UID: "7eec1800-06ec-47c6-bad2-dab46839c8b2") : secret "dns-default-metrics-tls" not found Jul 28 17:17:57.346382 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346379 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.346515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346400 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.346515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346419 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b67f5e-e899-4e00-b3ef-3529c95d7072-cert\") pod \"ingress-canary-4z9q8\" (UID: \"52b67f5e-e899-4e00-b3ef-3529c95d7072\") " pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.346515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346445 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eec1800-06ec-47c6-bad2-dab46839c8b2-tmp-dir\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.346515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346468 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cjv9\" (UniqueName: \"kubernetes.io/projected/7eec1800-06ec-47c6-bad2-dab46839c8b2-kube-api-access-9cjv9\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.346515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346492 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0989e1e-1e45-401f-b97b-c8ec8500e786-data-volume\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.346515 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.346499 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:17:57.346515 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.346508 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:17:57.346796 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346520 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v25k\" (UniqueName: \"kubernetes.io/projected/52b67f5e-e899-4e00-b3ef-3529c95d7072-kube-api-access-7v25k\") pod \"ingress-canary-4z9q8\" (UID: \"52b67f5e-e899-4e00-b3ef-3529c95d7072\") " pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.346796 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.346539 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:57.846527195 +0000 UTC m=+36.064568650 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : secret "insights-runtime-extractor-tls" not found Jul 28 17:17:57.346796 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.346571 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:57.846561685 +0000 UTC m=+36.064603137 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : configmap "kube-rbac-proxy" not found Jul 28 17:17:57.346796 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346595 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c0989e1e-1e45-401f-b97b-c8ec8500e786-crio-socket\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.346796 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346752 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eec1800-06ec-47c6-bad2-dab46839c8b2-tmp-dir\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.346796 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346784 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7eec1800-06ec-47c6-bad2-dab46839c8b2-config-volume\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.347033 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346808 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c0989e1e-1e45-401f-b97b-c8ec8500e786-data-volume\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.347033 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.346867 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c0989e1e-1e45-401f-b97b-c8ec8500e786-crio-socket\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.362642 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.362590 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46hkk\" (UniqueName: \"kubernetes.io/projected/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-api-access-46hkk\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.362642 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.362635 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cjv9\" (UniqueName: \"kubernetes.io/projected/7eec1800-06ec-47c6-bad2-dab46839c8b2-kube-api-access-9cjv9\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.439670 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.439650 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:57.447786 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.447764 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b67f5e-e899-4e00-b3ef-3529c95d7072-cert\") pod \"ingress-canary-4z9q8\" (UID: \"52b67f5e-e899-4e00-b3ef-3529c95d7072\") " pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.447851 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.447792 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v25k\" (UniqueName: \"kubernetes.io/projected/52b67f5e-e899-4e00-b3ef-3529c95d7072-kube-api-access-7v25k\") pod \"ingress-canary-4z9q8\" (UID: \"52b67f5e-e899-4e00-b3ef-3529c95d7072\") " pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.451089 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.450895 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/52b67f5e-e899-4e00-b3ef-3529c95d7072-cert\") pod \"ingress-canary-4z9q8\" (UID: \"52b67f5e-e899-4e00-b3ef-3529c95d7072\") " pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.456778 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.456760 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v25k\" (UniqueName: \"kubernetes.io/projected/52b67f5e-e899-4e00-b3ef-3529c95d7072-kube-api-access-7v25k\") pod \"ingress-canary-4z9q8\" (UID: \"52b67f5e-e899-4e00-b3ef-3529c95d7072\") " pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.462398 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.462377 2554 generic.go:356] "Generic (PLEG): container finished" podID="37e9f3f8-63cd-4635-9a06-d59209a1d68f" containerID="7273b210f66ef442a8e596e1be2efc61fd0bdcf30fafcfe97006bd810be2dedf" exitCode=0 Jul 28 17:17:57.462485 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.462408 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerDied","Data":"7273b210f66ef442a8e596e1be2efc61fd0bdcf30fafcfe97006bd810be2dedf"} Jul 28 17:17:57.508972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.508939 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:57.612992 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.612939 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4z9q8" Jul 28 17:17:57.790095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.789944 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4z9q8"] Jul 28 17:17:57.792951 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:17:57.792922 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52b67f5e_e899_4e00_b3ef_3529c95d7072.slice/crio-b71cbfbfd21dcec323348bef590812ace7d4cae4b3cbfc3fe020f9f660ea96a1 WatchSource:0}: Error finding container b71cbfbfd21dcec323348bef590812ace7d4cae4b3cbfc3fe020f9f660ea96a1: Status 404 returned error can't find the container with id b71cbfbfd21dcec323348bef590812ace7d4cae4b3cbfc3fe020f9f660ea96a1 Jul 28 17:17:57.852579 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.852554 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:57.852686 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.852587 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.852686 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:57.852612 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:57.852806 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.852702 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:17:57.852806 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.852767 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:58.852746566 +0000 UTC m=+37.070788009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : configmap "kube-rbac-proxy" not found Jul 28 17:17:57.852806 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.852705 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:57.852806 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.852783 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:17:57.853028 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.852829 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:58.852816474 +0000 UTC m=+37.070857914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : secret "insights-runtime-extractor-tls" not found Jul 28 17:17:57.853028 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:57.852842 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls podName:7eec1800-06ec-47c6-bad2-dab46839c8b2 nodeName:}" failed. No retries permitted until 2026-07-28 17:17:58.852836665 +0000 UTC m=+37.070878106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls") pod "dns-default-dzxzw" (UID: "7eec1800-06ec-47c6-bad2-dab46839c8b2") : secret "dns-default-metrics-tls" not found Jul 28 17:17:58.336011 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.335981 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:17:58.336011 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.335997 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:17:58.339368 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.339344 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:17:58.339515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.339349 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzphg\"" Jul 28 17:17:58.341132 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.341116 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:17:58.341225 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.341118 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:17:58.466384 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.466355 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4z9q8" event={"ID":"52b67f5e-e899-4e00-b3ef-3529c95d7072","Type":"ContainerStarted","Data":"b71cbfbfd21dcec323348bef590812ace7d4cae4b3cbfc3fe020f9f660ea96a1"} Jul 28 17:17:58.469864 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.469835 2554 generic.go:356] "Generic (PLEG): container finished" podID="37e9f3f8-63cd-4635-9a06-d59209a1d68f" containerID="47312290d422882e6357be764fe106c54561dcbf7cd0cb9c4c7a66854c25d9c0" exitCode=0 Jul 28 17:17:58.469981 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.469887 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerDied","Data":"47312290d422882e6357be764fe106c54561dcbf7cd0cb9c4c7a66854c25d9c0"} Jul 28 17:17:58.858927 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.858851 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:17:58.858927 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.858915 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:58.859142 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:58.858933 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:17:58.859142 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:58.858998 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:17:58.859142 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:58.859027 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:17:58.859142 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:58.859043 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:17:58.859142 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:58.859066 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls podName:7eec1800-06ec-47c6-bad2-dab46839c8b2 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:00.859045111 +0000 UTC m=+39.077086555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls") pod "dns-default-dzxzw" (UID: "7eec1800-06ec-47c6-bad2-dab46839c8b2") : secret "dns-default-metrics-tls" not found Jul 28 17:17:58.859142 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:58.859088 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:00.859077018 +0000 UTC m=+39.077118464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : secret "insights-runtime-extractor-tls" not found Jul 28 17:17:58.859142 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:17:58.859103 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:00.859094766 +0000 UTC m=+39.077136212 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : configmap "kube-rbac-proxy" not found Jul 28 17:17:59.476839 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:59.476800 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" event={"ID":"37e9f3f8-63cd-4635-9a06-d59209a1d68f","Type":"ContainerStarted","Data":"228bd3388e4c12e5a241c6cc7687cbaa948c9571baa7086b3cbc71978db4d2f4"} Jul 28 17:17:59.505567 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:59.505514 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7tgzl" podStartSLOduration=12.894415886 podStartE2EDuration="37.505501287s" totalImagesPullingTime="24.611085401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:59.503290706 +0000 UTC m=+37.721332170" watchObservedRunningTime="2026-07-28 17:17:59.505501287 +0000 UTC m=+37.723542757" Jul 28 17:17:59.642336 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:59.642302 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:59.743752 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:59.743673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:17:59.842715 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:17:59.842677 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:00.481029 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:00.480736 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4z9q8" event={"ID":"52b67f5e-e899-4e00-b3ef-3529c95d7072","Type":"ContainerStarted","Data":"adbba7b9b1ae4a4af9aadeef14b6a21da441cf65a6d637a5fbb1638f35dc43e0"} Jul 28 17:18:00.499162 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:00.499116 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4z9q8" podStartSLOduration=1.142570474 podStartE2EDuration="3.499101074s" totalImagesPullingTime="2.3565306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:00.498607729 +0000 UTC m=+38.716649192" watchObservedRunningTime="2026-07-28 17:18:00.499101074 +0000 UTC m=+38.717142528" Jul 28 17:18:00.874892 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:00.874843 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:00.874892 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:00.874889 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:00.875045 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:00.874908 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:00.875045 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:00.874964 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:18:00.875045 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:00.874977 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:18:00.875045 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:00.875018 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:04.875004792 +0000 UTC m=+43.093046238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : configmap "kube-rbac-proxy" not found Jul 28 17:18:00.875045 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:00.875025 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:18:00.875045 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:00.875032 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls podName:7eec1800-06ec-47c6-bad2-dab46839c8b2 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:04.875026015 +0000 UTC m=+43.093067459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls") pod "dns-default-dzxzw" (UID: "7eec1800-06ec-47c6-bad2-dab46839c8b2") : secret "dns-default-metrics-tls" not found Jul 28 17:18:00.875223 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:00.875058 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:04.875047888 +0000 UTC m=+43.093089330 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : secret "insights-runtime-extractor-tls" not found Jul 28 17:18:01.312029 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.311996 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:01.381264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.381241 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:01.395914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.395889 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm"] Jul 28 17:18:01.435715 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.435693 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm"] Jul 28 17:18:01.435804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.435792 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.438971 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.438956 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:18:01.439084 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.439063 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-phrc2\"" Jul 28 17:18:01.439130 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.439063 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 28 17:18:01.439176 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.439125 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 28 17:18:01.440387 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.440370 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 28 17:18:01.479129 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.479104 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxpqh\" (UniqueName: \"kubernetes.io/projected/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-kube-api-access-gxpqh\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.479221 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.479149 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-config\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.479221 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.479171 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.500448 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.500429 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc"] Jul 28 17:18:01.519305 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.519283 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc"] Jul 28 17:18:01.519373 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.519348 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.522003 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.521987 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 28 17:18:01.522178 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.522160 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 28 17:18:01.522178 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.522171 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 28 17:18:01.522338 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.522211 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:18:01.522338 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.522222 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jx7hc\"" Jul 28 17:18:01.580070 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.580028 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.580070 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.580065 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.580219 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.580084 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ljqk\" (UniqueName: \"kubernetes.io/projected/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-kube-api-access-2ljqk\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.580219 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.580125 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxpqh\" (UniqueName: \"kubernetes.io/projected/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-kube-api-access-gxpqh\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.580324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.580226 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-config\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.580324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.580270 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.580717 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.580698 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-config\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.584424 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.584403 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.591683 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.591663 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxpqh\" (UniqueName: \"kubernetes.io/projected/08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced-kube-api-access-gxpqh\") pod \"service-ca-operator-5d6c75f84f-gbhfm\" (UID: \"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.681405 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.681384 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.681482 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.681412 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.681482 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.681429 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ljqk\" (UniqueName: \"kubernetes.io/projected/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-kube-api-access-2ljqk\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.681887 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.681849 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.683182 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.683165 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.692326 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.691562 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ljqk\" (UniqueName: \"kubernetes.io/projected/ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3-kube-api-access-2ljqk\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc\" (UID: \"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.744700 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.744680 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" Jul 28 17:18:01.827179 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.827156 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" Jul 28 17:18:01.880135 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.880108 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm"] Jul 28 17:18:01.882716 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:01.882693 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08e7b8a7_10a8_44c8_9e4d_e5f2bdc2aced.slice/crio-8cbfb56bec64e87d86d3ca313a756619a727b3c959f0ea35ca6eea87842e85f3 WatchSource:0}: Error finding container 8cbfb56bec64e87d86d3ca313a756619a727b3c959f0ea35ca6eea87842e85f3: Status 404 returned error can't find the container with id 8cbfb56bec64e87d86d3ca313a756619a727b3c959f0ea35ca6eea87842e85f3 Jul 28 17:18:01.974482 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:01.974451 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc"] Jul 28 17:18:01.977324 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:01.977291 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce9c5b1d_7d82_40cc_a13f_3a78ade50ab3.slice/crio-0fcad4b4f617b31a6e856fa57ff6fd4b815c148b4c204f7a7688a7a24f6c7ec0 WatchSource:0}: Error finding container 0fcad4b4f617b31a6e856fa57ff6fd4b815c148b4c204f7a7688a7a24f6c7ec0: Status 404 returned error can't find the container with id 0fcad4b4f617b31a6e856fa57ff6fd4b815c148b4c204f7a7688a7a24f6c7ec0 Jul 28 17:18:02.254840 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.254816 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777"] Jul 28 17:18:02.293860 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.293843 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777"] Jul 28 17:18:02.293966 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.293952 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" Jul 28 17:18:02.296984 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.296965 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dgjsl\"" Jul 28 17:18:02.386017 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.385987 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmw8m\" (UniqueName: \"kubernetes.io/projected/83970fbd-be09-483e-8d0c-162d170fe472-kube-api-access-qmw8m\") pod \"network-check-source-7d88cb55d8-nb777\" (UID: \"83970fbd-be09-483e-8d0c-162d170fe472\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" Jul 28 17:18:02.476799 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.476769 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57ddcf98f7-qlz4z"] Jul 28 17:18:02.486373 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.486346 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmw8m\" (UniqueName: \"kubernetes.io/projected/83970fbd-be09-483e-8d0c-162d170fe472-kube-api-access-qmw8m\") pod \"network-check-source-7d88cb55d8-nb777\" (UID: \"83970fbd-be09-483e-8d0c-162d170fe472\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" Jul 28 17:18:02.499969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.499943 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmw8m\" (UniqueName: \"kubernetes.io/projected/83970fbd-be09-483e-8d0c-162d170fe472-kube-api-access-qmw8m\") pod \"network-check-source-7d88cb55d8-nb777\" (UID: \"83970fbd-be09-483e-8d0c-162d170fe472\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" Jul 28 17:18:02.503370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.503343 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" event={"ID":"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3","Type":"ContainerStarted","Data":"0fcad4b4f617b31a6e856fa57ff6fd4b815c148b4c204f7a7688a7a24f6c7ec0"} Jul 28 17:18:02.503731 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.503381 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57ddcf98f7-qlz4z"] Jul 28 17:18:02.503731 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.503404 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" event={"ID":"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced","Type":"ContainerStarted","Data":"8cbfb56bec64e87d86d3ca313a756619a727b3c959f0ea35ca6eea87842e85f3"} Jul 28 17:18:02.503731 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.503459 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.507233 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.507178 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 17:18:02.507964 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.507938 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4mv5p\"" Jul 28 17:18:02.508124 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.508110 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 17:18:02.508689 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.508671 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 17:18:02.514802 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.514782 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 17:18:02.587686 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.587659 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-trusted-ca\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.587777 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.587698 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-bound-sa-token\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.587777 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.587745 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-certificates\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.587777 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.587771 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-installation-pull-secrets\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.587939 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.587818 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-image-registry-private-configuration\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.587939 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.587849 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.587939 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.587868 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30353048-0b73-4b3b-b4a9-86da13a8bdc4-ca-trust-extracted\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.588080 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.588036 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrx2t\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-kube-api-access-lrx2t\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.603817 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.603793 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" Jul 28 17:18:02.688940 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.688903 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrx2t\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-kube-api-access-lrx2t\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689081 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.688973 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-trusted-ca\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689081 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.689000 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-bound-sa-token\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689081 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.689062 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-certificates\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.689089 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-installation-pull-secrets\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.689121 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-image-registry-private-configuration\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.689147 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.689175 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30353048-0b73-4b3b-b4a9-86da13a8bdc4-ca-trust-extracted\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.689676 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.689650 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30353048-0b73-4b3b-b4a9-86da13a8bdc4-ca-trust-extracted\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.691734 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.690926 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-trusted-ca\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.691734 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.691368 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-certificates\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.691734 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:02.691466 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:18:02.691734 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:02.691479 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57ddcf98f7-qlz4z: secret "image-registry-tls" not found Jul 28 17:18:02.691734 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:02.691531 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls podName:30353048-0b73-4b3b-b4a9-86da13a8bdc4 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:03.191512164 +0000 UTC m=+41.409553625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls") pod "image-registry-57ddcf98f7-qlz4z" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4") : secret "image-registry-tls" not found Jul 28 17:18:02.694102 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.694056 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-installation-pull-secrets\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.694689 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.694650 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-image-registry-private-configuration\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.703678 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.703605 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-bound-sa-token\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.707221 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.707204 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrx2t\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-kube-api-access-lrx2t\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:02.778573 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:02.778485 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777"] Jul 28 17:18:02.781756 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:02.781728 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83970fbd_be09_483e_8d0c_162d170fe472.slice/crio-f1f8be913a9a998d23589d2404f048a594a16ec40869da9bc7b7f649ec21af59 WatchSource:0}: Error finding container f1f8be913a9a998d23589d2404f048a594a16ec40869da9bc7b7f649ec21af59: Status 404 returned error can't find the container with id f1f8be913a9a998d23589d2404f048a594a16ec40869da9bc7b7f649ec21af59 Jul 28 17:18:03.193393 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:03.193305 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:03.193568 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:03.193499 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:18:03.193568 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:03.193524 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57ddcf98f7-qlz4z: secret "image-registry-tls" not found Jul 28 17:18:03.193686 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:03.193639 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls podName:30353048-0b73-4b3b-b4a9-86da13a8bdc4 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:04.193574566 +0000 UTC m=+42.411616021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls") pod "image-registry-57ddcf98f7-qlz4z" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4") : secret "image-registry-tls" not found Jul 28 17:18:03.460286 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:03.460203 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:03.492027 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:03.491989 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" event={"ID":"83970fbd-be09-483e-8d0c-162d170fe472","Type":"ContainerStarted","Data":"f1f8be913a9a998d23589d2404f048a594a16ec40869da9bc7b7f649ec21af59"} Jul 28 17:18:03.549498 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:03.549465 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:04.202515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:04.202469 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:04.202710 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.202608 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:18:04.202710 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.202635 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57ddcf98f7-qlz4z: secret "image-registry-tls" not found Jul 28 17:18:04.202710 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.202705 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls podName:30353048-0b73-4b3b-b4a9-86da13a8bdc4 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:06.202686607 +0000 UTC m=+44.420728070 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls") pod "image-registry-57ddcf98f7-qlz4z" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4") : secret "image-registry-tls" not found Jul 28 17:18:04.908472 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:04.908447 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:04.908487 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:04.908511 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.908589 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.908629 2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.908655 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls podName:7eec1800-06ec-47c6-bad2-dab46839c8b2 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:12.908635419 +0000 UTC m=+51.126676872 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls") pod "dns-default-dzxzw" (UID: "7eec1800-06ec-47c6-bad2-dab46839c8b2") : secret "dns-default-metrics-tls" not found Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.908656 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.908689 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:12.908671567 +0000 UTC m=+51.126713029 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : configmap "kube-rbac-proxy" not found Jul 28 17:18:04.908851 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:04.908712 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:12.908701492 +0000 UTC m=+51.126742961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : secret "insights-runtime-extractor-tls" not found Jul 28 17:18:05.498494 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:05.498454 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" event={"ID":"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced","Type":"ContainerStarted","Data":"cccdbce2fc2573db484100881e6ba619da9031ebba01bda08fbb14378b27e83e"} Jul 28 17:18:05.500184 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:05.500153 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" event={"ID":"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3","Type":"ContainerStarted","Data":"9e74ce69b6ee14ac0c5f5f3c650a8b3eeed46da3f0751689b789ed2cbc8bc2d7"} Jul 28 17:18:05.518542 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:05.518495 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" podStartSLOduration=1.567856498 podStartE2EDuration="4.518481651s" totalImagesPullingTime="2.950625153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:05.517973338 +0000 UTC m=+43.736014802" watchObservedRunningTime="2026-07-28 17:18:05.518481651 +0000 UTC m=+43.736523114" Jul 28 17:18:05.535220 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:05.535179 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" podStartSLOduration=1.6718362390000001 podStartE2EDuration="4.535165382s" totalImagesPullingTime="2.863329143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:05.534454399 +0000 UTC m=+43.752495864" watchObservedRunningTime="2026-07-28 17:18:05.535165382 +0000 UTC m=+43.753206845" Jul 28 17:18:06.218424 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:06.218389 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:06.218868 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:06.218581 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:18:06.218868 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:06.218606 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57ddcf98f7-qlz4z: secret "image-registry-tls" not found Jul 28 17:18:06.218868 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:06.218672 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls podName:30353048-0b73-4b3b-b4a9-86da13a8bdc4 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:10.218652855 +0000 UTC m=+48.436694300 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls") pod "image-registry-57ddcf98f7-qlz4z" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4") : secret "image-registry-tls" not found Jul 28 17:18:06.469271 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:06.469170 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:06.904387 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:06.904356 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:06.976237 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:06.976205 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:07.162900 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.162798 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:07.263234 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.263198 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:07.302680 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.302643 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd"] Jul 28 17:18:07.318715 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.318672 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd"] Jul 28 17:18:07.318901 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.318805 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" Jul 28 17:18:07.321699 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.321673 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 28 17:18:07.321699 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.321677 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 28 17:18:07.323290 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.323266 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-zm2rh\"" Jul 28 17:18:07.371092 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.371055 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:07.428649 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.428560 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ltdm\" (UniqueName: \"kubernetes.io/projected/858b4760-7ca2-47be-b8c5-fe46bdbbb932-kube-api-access-6ltdm\") pod \"migrator-6865c9967-q7fhd\" (UID: \"858b4760-7ca2-47be-b8c5-fe46bdbbb932\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" Jul 28 17:18:07.457540 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.457509 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:07.529373 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.529338 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ltdm\" (UniqueName: \"kubernetes.io/projected/858b4760-7ca2-47be-b8c5-fe46bdbbb932-kube-api-access-6ltdm\") pod \"migrator-6865c9967-q7fhd\" (UID: \"858b4760-7ca2-47be-b8c5-fe46bdbbb932\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" Jul 28 17:18:07.541798 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.541768 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ltdm\" (UniqueName: \"kubernetes.io/projected/858b4760-7ca2-47be-b8c5-fe46bdbbb932-kube-api-access-6ltdm\") pod \"migrator-6865c9967-q7fhd\" (UID: \"858b4760-7ca2-47be-b8c5-fe46bdbbb932\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" Jul 28 17:18:07.628072 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.628031 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" Jul 28 17:18:07.887916 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.887864 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd"] Jul 28 17:18:07.890499 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:07.890467 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod858b4760_7ca2_47be_b8c5_fe46bdbbb932.slice/crio-b65410648a4a7b1365686aabb2147f945b1539841d3647d1de0d1568d26d5af6 WatchSource:0}: Error finding container b65410648a4a7b1365686aabb2147f945b1539841d3647d1de0d1568d26d5af6: Status 404 returned error can't find the container with id b65410648a4a7b1365686aabb2147f945b1539841d3647d1de0d1568d26d5af6 Jul 28 17:18:07.913012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:07.912972 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:08.508624 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:08.508587 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" event={"ID":"83970fbd-be09-483e-8d0c-162d170fe472","Type":"ContainerStarted","Data":"0c18da5bae14d4a1a7db401f3e5979702101daeea3997d010aec11e9ba87e9fc"} Jul 28 17:18:08.509645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:08.509620 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" event={"ID":"858b4760-7ca2-47be-b8c5-fe46bdbbb932","Type":"ContainerStarted","Data":"b65410648a4a7b1365686aabb2147f945b1539841d3647d1de0d1568d26d5af6"} Jul 28 17:18:08.528554 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:08.528507 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-nb777" podStartSLOduration=1.5475214689999999 podStartE2EDuration="6.528493589s" totalImagesPullingTime="4.98097212s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:08.526959885 +0000 UTC m=+46.745001351" watchObservedRunningTime="2026-07-28 17:18:08.528493589 +0000 UTC m=+46.746535052" Jul 28 17:18:08.828795 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:08.828712 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:08.921302 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:08.921270 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:09.764652 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:09.764615 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:10.250890 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:10.250844 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:10.251028 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:10.250989 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:18:10.251028 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:10.251005 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57ddcf98f7-qlz4z: secret "image-registry-tls" not found Jul 28 17:18:10.251113 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:10.251057 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls podName:30353048-0b73-4b3b-b4a9-86da13a8bdc4 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:18.251040845 +0000 UTC m=+56.469082306 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls") pod "image-registry-57ddcf98f7-qlz4z" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4") : secret "image-registry-tls" not found Jul 28 17:18:10.517157 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:10.517079 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" event={"ID":"858b4760-7ca2-47be-b8c5-fe46bdbbb932","Type":"ContainerStarted","Data":"2f8144b936a8a76285f8e0317066b0c025d17e545f55156bec32f87b84e7b4db"} Jul 28 17:18:10.517157 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:10.517129 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" event={"ID":"858b4760-7ca2-47be-b8c5-fe46bdbbb932","Type":"ContainerStarted","Data":"5d5f70684d1aa6d90d99a221ce95d9718c6e40dfd805d8bfddf0d6e1332492af"} Jul 28 17:18:10.537869 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:10.537815 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-q7fhd" podStartSLOduration=1.812816583 podStartE2EDuration="3.53779851s" totalImagesPullingTime="1.724981927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:10.537290906 +0000 UTC m=+48.755332381" watchObservedRunningTime="2026-07-28 17:18:10.53779851 +0000 UTC m=+48.755839973" Jul 28 17:18:11.047299 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:11.047254 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:11.133740 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:11.133703 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:12.972844 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:12.972810 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:12.972844 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:12.972845 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:12.973268 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:12.972915 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:12.973268 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:12.973000 2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:18:12.973268 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:12.973016 2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:18:12.973268 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:12.973048 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls podName:7eec1800-06ec-47c6-bad2-dab46839c8b2 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:28.97303606 +0000 UTC m=+67.191077502 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls") pod "dns-default-dzxzw" (UID: "7eec1800-06ec-47c6-bad2-dab46839c8b2") : secret "dns-default-metrics-tls" not found Jul 28 17:18:12.973268 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:12.973067 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls podName:c0989e1e-1e45-401f-b97b-c8ec8500e786 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:28.973055491 +0000 UTC m=+67.191096932 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rflm7" (UID: "c0989e1e-1e45-401f-b97b-c8ec8500e786") : secret "insights-runtime-extractor-tls" not found Jul 28 17:18:12.973452 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:12.973300 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c0989e1e-1e45-401f-b97b-c8ec8500e786-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:13.377757 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:13.377730 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:14.544079 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:14.544031 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:14.657725 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:14.657683 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:15.274333 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:15.274304 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:15.529203 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:15.529116 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:15.630008 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:15.629980 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:15.729441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:15.729415 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:15.793510 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:15.793438 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:15.971423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:15.971386 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:16.768903 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:16.768862 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:17.364469 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:17.364430 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:17.562222 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:17.562145 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:17.770509 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:17.770481 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:18.162882 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:18.162794 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:18.312675 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:18.312646 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") pod \"image-registry-57ddcf98f7-qlz4z\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:18.312808 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:18.312781 2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:18:18.312808 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:18.312793 2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57ddcf98f7-qlz4z: secret "image-registry-tls" not found Jul 28 17:18:18.312913 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:18.312840 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls podName:30353048-0b73-4b3b-b4a9-86da13a8bdc4 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:34.31282712 +0000 UTC m=+72.530868566 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls") pod "image-registry-57ddcf98f7-qlz4z" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4") : secret "image-registry-tls" not found Jul 28 17:18:18.569957 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:18.569929 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:18.894205 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:18.894146 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:18.998506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:18.998482 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:19.581246 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:19.581211 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:19.681975 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:19.681945 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:19.782835 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:19.782805 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:19.841643 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:19.841572 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:20.165542 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:20.165476 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:20.768832 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:20.768804 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:21.046378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:21.046323 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:21.173657 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:21.173634 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:21.563085 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:21.563062 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:21.763807 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:21.763785 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:22.204963 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:22.204942 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:22.400303 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:22.400276 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:22.459592 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:22.459536 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fjw6m" Jul 28 17:18:22.774138 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:22.774115 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:22.979736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:22.979708 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:23.566340 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:23.566313 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:23.773451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:23.773424 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:24.172304 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:24.172278 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:24.303399 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:24.303369 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:24.412133 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:24.412101 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:24.672989 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:24.672954 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:24.773435 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:24.773413 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.005914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.005888 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.061432 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.061409 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.184943 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.184924 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.372207 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.372155 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.471444 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.471427 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.688891 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.688822 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.790434 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.790413 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.889983 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.889962 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:25.989557 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:25.989501 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:26.091950 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:26.091929 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:26.192499 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:26.192471 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:26.289944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:26.289919 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:26.393436 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:26.393411 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:26.770595 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:26.770566 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:27.978159 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:27.978129 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:18:27.981358 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:27.981339 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:18:27.990808 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:27.990790 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7809890-59e0-45c7-9286-5f0fe38a4a64-metrics-certs\") pod \"network-metrics-daemon-rbkjh\" (UID: \"a7809890-59e0-45c7-9286-5f0fe38a4a64\") " pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:18:28.079009 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.078985 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:18:28.079095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.079018 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:18:28.081187 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.081170 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lttmp\" (UniqueName: \"kubernetes.io/projected/8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c-kube-api-access-lttmp\") pod \"network-check-target-fl4xc\" (UID: \"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c\") " pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:18:28.082432 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.082411 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:18:28.091790 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.091773 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fda79549-f59d-4702-b9a4-09416b782d4c-original-pull-secret\") pod \"global-pull-secret-syncer-xq4v4\" (UID: \"fda79549-f59d-4702-b9a4-09416b782d4c\") " pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:18:28.242677 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.242626 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vvwzr\"" Jul 28 17:18:28.250144 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.250130 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rbkjh" Jul 28 17:18:28.346652 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.346576 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xq4v4" Jul 28 17:18:28.353947 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.353928 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qzphg\"" Jul 28 17:18:28.361545 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.361520 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:18:28.405148 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.405117 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rbkjh"] Jul 28 17:18:28.477577 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.477552 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xq4v4"] Jul 28 17:18:28.479643 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:28.479619 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda79549_f59d_4702_b9a4_09416b782d4c.slice/crio-f4be2d47c9beb637ff14431864e84e918e505f6fca4f60a89e8b0efade58c7fb WatchSource:0}: Error finding container f4be2d47c9beb637ff14431864e84e918e505f6fca4f60a89e8b0efade58c7fb: Status 404 returned error can't find the container with id f4be2d47c9beb637ff14431864e84e918e505f6fca4f60a89e8b0efade58c7fb Jul 28 17:18:28.498255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.498236 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fl4xc"] Jul 28 17:18:28.499566 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:28.499545 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cba4b44_e7b9_43de_bdf8_5d4d936c6c9c.slice/crio-4724abbfb0e18e241cf9516c067cc60320132f71bd66fed2aa0f12ac23dbc6db WatchSource:0}: Error finding container 4724abbfb0e18e241cf9516c067cc60320132f71bd66fed2aa0f12ac23dbc6db: Status 404 returned error can't find the container with id 4724abbfb0e18e241cf9516c067cc60320132f71bd66fed2aa0f12ac23dbc6db Jul 28 17:18:28.567783 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.567755 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fl4xc" event={"ID":"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c","Type":"ContainerStarted","Data":"8e1236f6bfbbad9cef992a4af25707cba56a1247e9b19809c773fbc0a5df1bad"} Jul 28 17:18:28.567902 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.567809 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fl4xc" event={"ID":"8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c","Type":"ContainerStarted","Data":"4724abbfb0e18e241cf9516c067cc60320132f71bd66fed2aa0f12ac23dbc6db"} Jul 28 17:18:28.567902 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.567864 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:18:28.569005 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.568984 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xq4v4" event={"ID":"fda79549-f59d-4702-b9a4-09416b782d4c","Type":"ContainerStarted","Data":"f4be2d47c9beb637ff14431864e84e918e505f6fca4f60a89e8b0efade58c7fb"} Jul 28 17:18:28.569979 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.569959 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rbkjh" event={"ID":"a7809890-59e0-45c7-9286-5f0fe38a4a64","Type":"ContainerStarted","Data":"3fa9f9bf0f25a0ebfe37b446669a16dec948fcbd476f089f3906f9f537154469"} Jul 28 17:18:28.587024 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.586986 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fl4xc" podStartSLOduration=66.586973277 podStartE2EDuration="1m6.586973277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:28.586334641 +0000 UTC m=+66.804376108" watchObservedRunningTime="2026-07-28 17:18:28.586973277 +0000 UTC m=+66.805014786" Jul 28 17:18:28.986188 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.986111 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:28.986577 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.986189 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:28.989175 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.989089 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7eec1800-06ec-47c6-bad2-dab46839c8b2-metrics-tls\") pod \"dns-default-dzxzw\" (UID: \"7eec1800-06ec-47c6-bad2-dab46839c8b2\") " pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:28.989175 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.989166 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c0989e1e-1e45-401f-b97b-c8ec8500e786-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rflm7\" (UID: \"c0989e1e-1e45-401f-b97b-c8ec8500e786\") " pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:28.998889 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:28.998847 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qjb4d\"" Jul 28 17:18:29.006114 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.006090 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:29.006227 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.006201 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57ddcf98f7-qlz4z"] Jul 28 17:18:29.006423 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:29.006387 2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" podUID="30353048-0b73-4b3b-b4a9-86da13a8bdc4" Jul 28 17:18:29.023112 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.023089 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mc5b9\"" Jul 28 17:18:29.030578 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.030559 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rflm7" Jul 28 17:18:29.050721 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.050691 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-lh9t4"] Jul 28 17:18:29.052826 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.052805 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lh9t4" Jul 28 17:18:29.055957 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.055939 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 17:18:29.056062 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.056022 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7dcbm\"" Jul 28 17:18:29.056269 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.056252 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 17:18:29.068690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.068655 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lh9t4"] Jul 28 17:18:29.110612 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.110550 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:29.170112 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.170081 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dzxzw"] Jul 28 17:18:29.173706 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:29.173674 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eec1800_06ec_47c6_bad2_dab46839c8b2.slice/crio-dde82704073b0ce1e4fdccbf07cf64e39cc993917bd15a0e1a850568ca59808f WatchSource:0}: Error finding container dde82704073b0ce1e4fdccbf07cf64e39cc993917bd15a0e1a850568ca59808f: Status 404 returned error can't find the container with id dde82704073b0ce1e4fdccbf07cf64e39cc993917bd15a0e1a850568ca59808f Jul 28 17:18:29.184574 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.184503 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rflm7"] Jul 28 17:18:29.187516 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:29.187486 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0989e1e_1e45_401f_b97b_c8ec8500e786.slice/crio-86afa11ce27b8c8aebc0ace75434585f93f48787adc118a09a3a813fb384d337 WatchSource:0}: Error finding container 86afa11ce27b8c8aebc0ace75434585f93f48787adc118a09a3a813fb384d337: Status 404 returned error can't find the container with id 86afa11ce27b8c8aebc0ace75434585f93f48787adc118a09a3a813fb384d337 Jul 28 17:18:29.188578 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.188545 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5rpf\" (UniqueName: \"kubernetes.io/projected/3a95e312-298d-4ad1-b495-8f8d98a7ccb4-kube-api-access-p5rpf\") pod \"downloads-7c97dc4555-lh9t4\" (UID: \"3a95e312-298d-4ad1-b495-8f8d98a7ccb4\") " pod="openshift-console/downloads-7c97dc4555-lh9t4" Jul 28 17:18:29.192449 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.192423 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:29.269777 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.269751 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:29.289416 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.289384 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5rpf\" (UniqueName: \"kubernetes.io/projected/3a95e312-298d-4ad1-b495-8f8d98a7ccb4-kube-api-access-p5rpf\") pod \"downloads-7c97dc4555-lh9t4\" (UID: \"3a95e312-298d-4ad1-b495-8f8d98a7ccb4\") " pod="openshift-console/downloads-7c97dc4555-lh9t4" Jul 28 17:18:29.301358 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.301333 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5rpf\" (UniqueName: \"kubernetes.io/projected/3a95e312-298d-4ad1-b495-8f8d98a7ccb4-kube-api-access-p5rpf\") pod \"downloads-7c97dc4555-lh9t4\" (UID: \"3a95e312-298d-4ad1-b495-8f8d98a7ccb4\") " pod="openshift-console/downloads-7c97dc4555-lh9t4" Jul 28 17:18:29.367376 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.367313 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:29.368348 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.368159 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-lh9t4" Jul 28 17:18:29.462894 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.462846 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:29.562357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.562327 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:29.575838 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.575806 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rflm7" event={"ID":"c0989e1e-1e45-401f-b97b-c8ec8500e786","Type":"ContainerStarted","Data":"86afa11ce27b8c8aebc0ace75434585f93f48787adc118a09a3a813fb384d337"} Jul 28 17:18:29.577291 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.577193 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzxzw" event={"ID":"7eec1800-06ec-47c6-bad2-dab46839c8b2","Type":"ContainerStarted","Data":"dde82704073b0ce1e4fdccbf07cf64e39cc993917bd15a0e1a850568ca59808f"} Jul 28 17:18:29.577291 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.577212 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:29.587936 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.586421 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:29.625718 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.625649 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:29.683936 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.683842 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-lh9t4"] Jul 28 17:18:29.688799 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:29.688765 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a95e312_298d_4ad1_b495_8f8d98a7ccb4.slice/crio-094e6e3e17d251c63d5e0056f448670b2a928f2e52d43ac0417d3bbef692f952 WatchSource:0}: Error finding container 094e6e3e17d251c63d5e0056f448670b2a928f2e52d43ac0417d3bbef692f952: Status 404 returned error can't find the container with id 094e6e3e17d251c63d5e0056f448670b2a928f2e52d43ac0417d3bbef692f952 Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.692081 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-bound-sa-token\") pod \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.692131 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-certificates\") pod \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.692173 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-image-registry-private-configuration\") pod \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.692201 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-installation-pull-secrets\") pod \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.692257 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30353048-0b73-4b3b-b4a9-86da13a8bdc4-ca-trust-extracted\") pod \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.692289 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-trusted-ca\") pod \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.692315 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrx2t\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-kube-api-access-lrx2t\") pod \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\" (UID: \"30353048-0b73-4b3b-b4a9-86da13a8bdc4\") " Jul 28 17:18:29.693658 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.693316 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "30353048-0b73-4b3b-b4a9-86da13a8bdc4" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:18:29.694215 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.693674 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30353048-0b73-4b3b-b4a9-86da13a8bdc4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "30353048-0b73-4b3b-b4a9-86da13a8bdc4" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:18:29.694215 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.693775 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "30353048-0b73-4b3b-b4a9-86da13a8bdc4" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:18:29.697447 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.697409 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "30353048-0b73-4b3b-b4a9-86da13a8bdc4" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:18:29.697664 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.697642 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "30353048-0b73-4b3b-b4a9-86da13a8bdc4" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:18:29.698101 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.698073 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "30353048-0b73-4b3b-b4a9-86da13a8bdc4" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:18:29.699299 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.699264 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-kube-api-access-lrx2t" (OuterVolumeSpecName: "kube-api-access-lrx2t") pod "30353048-0b73-4b3b-b4a9-86da13a8bdc4" (UID: "30353048-0b73-4b3b-b4a9-86da13a8bdc4"). InnerVolumeSpecName "kube-api-access-lrx2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:18:29.794441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.794283 2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-bound-sa-token\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:29.794441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.794313 2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-certificates\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:29.794441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.794328 2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-image-registry-private-configuration\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:29.794441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.794342 2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/30353048-0b73-4b3b-b4a9-86da13a8bdc4-installation-pull-secrets\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:29.794441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.794356 2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/30353048-0b73-4b3b-b4a9-86da13a8bdc4-ca-trust-extracted\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:29.794441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.794369 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/30353048-0b73-4b3b-b4a9-86da13a8bdc4-trusted-ca\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:29.794441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:29.794381 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrx2t\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-kube-api-access-lrx2t\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:30.593229 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.593183 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lh9t4" event={"ID":"3a95e312-298d-4ad1-b495-8f8d98a7ccb4","Type":"ContainerStarted","Data":"094e6e3e17d251c63d5e0056f448670b2a928f2e52d43ac0417d3bbef692f952"} Jul 28 17:18:30.597825 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.596784 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57ddcf98f7-qlz4z" Jul 28 17:18:30.597825 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.597682 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rbkjh" event={"ID":"a7809890-59e0-45c7-9286-5f0fe38a4a64","Type":"ContainerStarted","Data":"4506df193d4dc6cb58af8f9122293faf2f8003d18c04eac002241977effffde7"} Jul 28 17:18:30.597825 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.597722 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rbkjh" event={"ID":"a7809890-59e0-45c7-9286-5f0fe38a4a64","Type":"ContainerStarted","Data":"702812ad43fbedd3aac4941d8c632654d4769a61a5ff8774392ef2a2055a7bac"} Jul 28 17:18:30.650310 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.649797 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rbkjh" podStartSLOduration=67.496609245 podStartE2EDuration="1m8.64977876s" totalImagesPullingTime="1.153169515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:30.624228841 +0000 UTC m=+68.842270305" watchObservedRunningTime="2026-07-28 17:18:30.64977876 +0000 UTC m=+68.867820226" Jul 28 17:18:30.661141 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.660979 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57ddcf98f7-qlz4z"] Jul 28 17:18:30.665715 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.665654 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57ddcf98f7-qlz4z"] Jul 28 17:18:30.731781 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.731754 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:30.804681 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.804655 2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/30353048-0b73-4b3b-b4a9-86da13a8bdc4-registry-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:18:30.832785 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.832692 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:30.899491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:30.899412 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:32.338470 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:32.338271 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30353048-0b73-4b3b-b4a9-86da13a8bdc4" path="/var/lib/kubelet/pods/30353048-0b73-4b3b-b4a9-86da13a8bdc4/volumes" Jul 28 17:18:32.606543 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:32.606456 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzxzw" event={"ID":"7eec1800-06ec-47c6-bad2-dab46839c8b2","Type":"ContainerStarted","Data":"fc5dc2be9c68f57687bc8224a1dc0a1f64eba94464a2a5bdd34c0c2bb1684cf1"} Jul 28 17:18:32.606543 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:32.606517 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dzxzw" event={"ID":"7eec1800-06ec-47c6-bad2-dab46839c8b2","Type":"ContainerStarted","Data":"84fd3ec3979aedd9f94999ed1353d6c5104f7dc64bd03b123a7d4b7b6e3e9f6c"} Jul 28 17:18:32.606736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:32.606570 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:32.627806 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:32.627751 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dzxzw" podStartSLOduration=32.865818261 podStartE2EDuration="35.627732552s" totalImagesPullingTime="2.761914291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:32.62654946 +0000 UTC m=+70.844590922" watchObservedRunningTime="2026-07-28 17:18:32.627732552 +0000 UTC m=+70.845774016" Jul 28 17:18:32.891434 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:32.891350 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:32.987491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:32.987453 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:34.085203 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.085174 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5bcb58489-bjzm9"] Jul 28 17:18:34.172758 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.172699 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bcb58489-bjzm9"] Jul 28 17:18:34.172942 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.172810 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.177156 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.177130 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 17:18:34.177490 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.177467 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 17:18:34.179338 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.178953 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 17:18:34.179338 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.178977 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 17:18:34.179672 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.179560 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-66zv9\"" Jul 28 17:18:34.179672 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.179560 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 17:18:34.332589 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.332552 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-config\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.332752 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.332616 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-serving-cert\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.332752 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.332672 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-oauth-config\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.332752 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.332722 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nw4b\" (UniqueName: \"kubernetes.io/projected/6cee8893-4223-4bf6-9653-f7bfbfab1464-kube-api-access-9nw4b\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.333002 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.332750 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-service-ca\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.333002 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.332792 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-oauth-serving-cert\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.433610 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.433531 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-serving-cert\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.433610 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.433590 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-oauth-config\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.433827 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.433629 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nw4b\" (UniqueName: \"kubernetes.io/projected/6cee8893-4223-4bf6-9653-f7bfbfab1464-kube-api-access-9nw4b\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.433827 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.433663 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-service-ca\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.433827 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.433709 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-oauth-serving-cert\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.433827 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.433754 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-config\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.434752 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.434721 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-oauth-serving-cert\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.434887 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.434721 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-service-ca\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.434961 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.434915 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-config\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.436595 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.436550 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-oauth-config\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.437144 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.437124 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-serving-cert\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.445451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.445427 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nw4b\" (UniqueName: \"kubernetes.io/projected/6cee8893-4223-4bf6-9653-f7bfbfab1464-kube-api-access-9nw4b\") pod \"console-5bcb58489-bjzm9\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:34.484406 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:34.484379 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:35.819420 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:35.819334 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:35.884347 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:35.884310 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:35.945070 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:35.945041 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:36.007575 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:36.007546 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:36.154641 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:36.154614 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:36.953327 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:36.953203 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5bcb58489-bjzm9"] Jul 28 17:18:36.956395 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:36.956371 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cee8893_4223_4bf6_9653_f7bfbfab1464.slice/crio-f8180615f26d6641e94a43423d7beebfb56747f680658a490c661e65685b3376 WatchSource:0}: Error finding container f8180615f26d6641e94a43423d7beebfb56747f680658a490c661e65685b3376: Status 404 returned error can't find the container with id f8180615f26d6641e94a43423d7beebfb56747f680658a490c661e65685b3376 Jul 28 17:18:37.627903 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:37.627519 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xq4v4" event={"ID":"fda79549-f59d-4702-b9a4-09416b782d4c","Type":"ContainerStarted","Data":"2d7770f0e9499580074094b4533e070405193018ca1a874148996e148126e2c2"} Jul 28 17:18:37.629910 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:37.629830 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rflm7" event={"ID":"c0989e1e-1e45-401f-b97b-c8ec8500e786","Type":"ContainerStarted","Data":"c2aaace4fb4e59d0b4a61114c9a7a640e1d78c58821740a2a2812011c04333b0"} Jul 28 17:18:37.631233 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:37.631186 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcb58489-bjzm9" event={"ID":"6cee8893-4223-4bf6-9653-f7bfbfab1464","Type":"ContainerStarted","Data":"f8180615f26d6641e94a43423d7beebfb56747f680658a490c661e65685b3376"} Jul 28 17:18:37.646435 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:37.646385 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xq4v4" podStartSLOduration=67.330634207 podStartE2EDuration="1m15.646366768s" totalImagesPullingTime="8.315732561s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:37.646005379 +0000 UTC m=+75.864046843" watchObservedRunningTime="2026-07-28 17:18:37.646366768 +0000 UTC m=+75.864408233" Jul 28 17:18:39.208415 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:39.208342 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:39.543829 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:39.543792 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:39.882213 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:39.882141 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:39.966131 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:39.966090 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.088992 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.088953 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.333676 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.333637 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.415484 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.415377 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.428687 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.428489 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-67wqk"] Jul 28 17:18:41.449755 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.449732 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.456088 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.456066 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 17:18:41.456245 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.456224 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 17:18:41.457171 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.456679 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v97v9\"" Jul 28 17:18:41.457171 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.456989 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 17:18:41.457827 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.457809 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 17:18:41.458428 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.458407 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 17:18:41.460694 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.460646 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 17:18:41.499162 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.499119 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.593101 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593069 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-sys\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593128 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593205 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-metrics-client-ca\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593254 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-textfile\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593446 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593272 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-wtmp\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593446 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593296 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-tls\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593446 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593353 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-root\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593446 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593388 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-accelerators-collector-config\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.593446 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.593424 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhlbx\" (UniqueName: \"kubernetes.io/projected/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-kube-api-access-mhlbx\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.600713 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.600673 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.694180 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694097 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-sys\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694180 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694145 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694361 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694223 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-sys\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694361 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694275 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-metrics-client-ca\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694361 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694322 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-textfile\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694361 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694350 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-wtmp\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694533 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694387 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-tls\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694533 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694425 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-root\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694533 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694459 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-accelerators-collector-config\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694533 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694494 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhlbx\" (UniqueName: \"kubernetes.io/projected/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-kube-api-access-mhlbx\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.694664 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694582 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-wtmp\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.695108 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.694784 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-root\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.695108 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:41.694841 2554 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 17:18:41.695108 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:18:41.694918 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-tls podName:4b94e0cb-3eb3-4c81-aeba-0117020c9de0 nodeName:}" failed. No retries permitted until 2026-07-28 17:18:42.194899653 +0000 UTC m=+80.412941103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-tls") pod "node-exporter-67wqk" (UID: "4b94e0cb-3eb3-4c81-aeba-0117020c9de0") : secret "node-exporter-tls" not found Jul 28 17:18:41.695108 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.695050 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-textfile\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.695389 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.695155 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-metrics-client-ca\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.695441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.695381 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-accelerators-collector-config\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.697463 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.697420 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.701400 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.701295 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.705446 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.705424 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhlbx\" (UniqueName: \"kubernetes.io/projected/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-kube-api-access-mhlbx\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:41.810945 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.810905 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:41.926263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:41.926230 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:42.074900 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.074849 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:42.178428 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.178391 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:42.199494 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.199468 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-tls\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:42.201811 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.201783 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b94e0cb-3eb3-4c81-aeba-0117020c9de0-node-exporter-tls\") pod \"node-exporter-67wqk\" (UID: \"4b94e0cb-3eb3-4c81-aeba-0117020c9de0\") " pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:42.362106 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.362037 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-67wqk" Jul 28 17:18:42.375707 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.375674 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:42.478921 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.478886 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:18:42.479572 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.479538 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:42.512914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.512866 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:18:42.513076 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.513014 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.516202 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.516182 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 17:18:42.516511 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.516287 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qw8dm\"" Jul 28 17:18:42.516511 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.516371 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 17:18:42.517055 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.516929 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 17:18:42.517055 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.516941 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 17:18:42.517055 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.516948 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 17:18:42.517055 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.516941 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 17:18:42.517602 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.517404 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 17:18:42.519012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.518333 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 17:18:42.519012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.518427 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 17:18:42.606505 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.606478 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6897794bf-wjbvf"] Jul 28 17:18:42.608407 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608385 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608541 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608426 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608541 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608464 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608541 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608491 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608541 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608525 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608783 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608546 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-web-config\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608783 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608574 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608783 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608629 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4kwh\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-kube-api-access-b4kwh\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608783 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608688 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608783 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608718 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608993 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608782 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-config-out\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608993 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-config-volume\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.608993 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.608866 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.619859 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.619808 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6897794bf-wjbvf"] Jul 28 17:18:42.619982 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.619929 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.620051 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.619989 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dzxzw" Jul 28 17:18:42.628735 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.628465 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 17:18:42.709393 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709364 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.709567 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709402 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-serving-cert\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.709567 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709431 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-oauth-config\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.709567 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709479 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.709737 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709629 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-service-ca\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.709737 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709684 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.709737 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709719 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.709910 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709742 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-web-config\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.709910 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709791 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.709910 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709824 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4kwh\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-kube-api-access-b4kwh\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.709910 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709865 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-oauth-serving-cert\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.710111 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709909 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.710111 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709936 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.710111 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709965 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-config-out\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.710111 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.709989 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-trusted-ca-bundle\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.710111 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.710039 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-config-volume\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.710111 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.710087 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-config\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.710410 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.710115 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cjfs\" (UniqueName: \"kubernetes.io/projected/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-kube-api-access-7cjfs\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.710410 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.710147 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.710410 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.710182 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.711946 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.711437 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.711946 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.711817 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.711946 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.711897 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.713165 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.713137 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.713563 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.713538 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.713929 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.713904 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-web-config\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.714720 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.714683 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.715427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.715337 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-config-volume\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.715427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.715351 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-config-out\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.715427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.715387 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.715618 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.715454 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.716535 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.716516 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.719545 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.719524 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4kwh\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-kube-api-access-b4kwh\") pod \"alertmanager-main-0\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.727233 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.727209 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:42.811133 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811101 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-config\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.811295 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811142 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cjfs\" (UniqueName: \"kubernetes.io/projected/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-kube-api-access-7cjfs\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.811295 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811182 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-serving-cert\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.811295 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811247 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-oauth-config\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.811295 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811286 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-service-ca\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.811509 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811356 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-oauth-serving-cert\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.811509 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811395 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-trusted-ca-bundle\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.811856 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.811832 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-config\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.812062 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.812004 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-service-ca\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.812177 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.812124 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-oauth-serving-cert\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.812309 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.812287 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-trusted-ca-bundle\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.814051 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.814033 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-serving-cert\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.814157 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.814064 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-oauth-config\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.821656 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.821636 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cjfs\" (UniqueName: \"kubernetes.io/projected/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-kube-api-access-7cjfs\") pod \"console-6897794bf-wjbvf\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:42.825351 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.825331 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:18:42.931796 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:42.931727 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:44.348644 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.348504 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:44.449114 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.449085 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:44.459317 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.459296 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-55557df95f-8vrp5"] Jul 28 17:18:44.463769 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.463751 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.466603 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.466570 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 17:18:44.466721 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.466691 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-vhwq8\"" Jul 28 17:18:44.466801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.466739 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 17:18:44.466860 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.466822 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 17:18:44.467069 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.467053 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-23hcnvg9l3bqv\"" Jul 28 17:18:44.467123 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.467093 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 17:18:44.468089 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.468061 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 17:18:44.474688 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.474300 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55557df95f-8vrp5"] Jul 28 17:18:44.513434 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.513404 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:44.625222 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625153 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68b16455-4ea2-4be1-bfef-56a619605594-metrics-client-ca\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.625222 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625202 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.625408 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625233 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.625408 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625265 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.625408 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625314 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.625408 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625339 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-grpc-tls\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.625408 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625385 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-tls\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.625583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.625451 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-989nn\" (UniqueName: \"kubernetes.io/projected/68b16455-4ea2-4be1-bfef-56a619605594-kube-api-access-989nn\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.652038 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.651999 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:44.726079 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726051 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.726212 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726096 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.726212 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726130 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.726212 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726177 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.726212 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726204 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-grpc-tls\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.726417 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726231 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-tls\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.726417 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726255 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-989nn\" (UniqueName: \"kubernetes.io/projected/68b16455-4ea2-4be1-bfef-56a619605594-kube-api-access-989nn\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.726417 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.726311 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68b16455-4ea2-4be1-bfef-56a619605594-metrics-client-ca\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.727063 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.727042 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68b16455-4ea2-4be1-bfef-56a619605594-metrics-client-ca\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.729287 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.729250 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.729747 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.729725 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.729888 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.729834 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.729954 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.729937 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.730386 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.730362 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-grpc-tls\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.730507 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.730482 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/68b16455-4ea2-4be1-bfef-56a619605594-secret-thanos-querier-tls\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.735778 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.735753 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-989nn\" (UniqueName: \"kubernetes.io/projected/68b16455-4ea2-4be1-bfef-56a619605594-kube-api-access-989nn\") pod \"thanos-querier-55557df95f-8vrp5\" (UID: \"68b16455-4ea2-4be1-bfef-56a619605594\") " pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.776748 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.776719 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:18:44.898637 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.898570 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:45.000022 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:44.999985 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:45.685969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.685936 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt"] Jul 28 17:18:45.689077 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.689059 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.693508 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.693481 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 17:18:45.693616 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.693541 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4ctrtge67r01g\"" Jul 28 17:18:45.693616 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.693558 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 17:18:45.693616 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.693486 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 17:18:45.693774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.693652 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 17:18:45.693774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.693564 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-54hq2\"" Jul 28 17:18:45.700714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.700695 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt"] Jul 28 17:18:45.834256 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.834225 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-client-ca-bundle\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.834256 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.834261 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpclg\" (UniqueName: \"kubernetes.io/projected/432e2472-c9a3-479e-b840-f7202a4483cc-kube-api-access-gpclg\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.834491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.834289 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432e2472-c9a3-479e-b840-f7202a4483cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.834491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.834317 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-secret-metrics-server-client-certs\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.834491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.834369 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/432e2472-c9a3-479e-b840-f7202a4483cc-metrics-server-audit-profiles\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.834491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.834437 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-secret-metrics-server-tls\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.834491 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.834470 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/432e2472-c9a3-479e-b840-f7202a4483cc-audit-log\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.935618 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.935588 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-client-ca-bundle\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.935774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.935630 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpclg\" (UniqueName: \"kubernetes.io/projected/432e2472-c9a3-479e-b840-f7202a4483cc-kube-api-access-gpclg\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.935774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.935660 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432e2472-c9a3-479e-b840-f7202a4483cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.935774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.935690 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-secret-metrics-server-client-certs\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.935774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.935713 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/432e2472-c9a3-479e-b840-f7202a4483cc-metrics-server-audit-profiles\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.935774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.935759 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-secret-metrics-server-tls\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.936081 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.935804 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/432e2472-c9a3-479e-b840-f7202a4483cc-audit-log\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.936287 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.936241 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/432e2472-c9a3-479e-b840-f7202a4483cc-audit-log\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.936563 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.936535 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/432e2472-c9a3-479e-b840-f7202a4483cc-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.936844 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.936814 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/432e2472-c9a3-479e-b840-f7202a4483cc-metrics-server-audit-profiles\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.938429 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.938408 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-secret-metrics-server-client-certs\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.938573 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.938554 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-client-ca-bundle\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.938670 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.938650 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/432e2472-c9a3-479e-b840-f7202a4483cc-secret-metrics-server-tls\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:45.945308 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:45.945264 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpclg\" (UniqueName: \"kubernetes.io/projected/432e2472-c9a3-479e-b840-f7202a4483cc-kube-api-access-gpclg\") pod \"metrics-server-5d5b5d9d67-5ddtt\" (UID: \"432e2472-c9a3-479e-b840-f7202a4483cc\") " pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:46.000048 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:46.000026 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:18:46.105235 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:46.105206 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:46.174547 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:46.174521 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:46.767009 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:46.766948 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:47.628218 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.628186 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:18:47.629768 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.629728 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:47.635018 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.634994 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.637788 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.637761 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 17:18:47.637937 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.637892 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 17:18:47.638354 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.638329 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2sdfb\"" Jul 28 17:18:47.638450 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.638378 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 17:18:47.638450 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.638385 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 17:18:47.638450 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.638417 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 17:18:47.638681 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.638664 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 17:18:47.639735 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.639601 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 17:18:47.639735 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.639625 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 17:18:47.639735 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.639640 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 17:18:47.639958 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.639933 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 17:18:47.640177 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.640161 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 17:18:47.640262 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.640189 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 17:18:47.640262 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.640237 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 17:18:47.640544 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.640484 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5i3tmhjt30lb\"" Jul 28 17:18:47.641655 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.641635 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 17:18:47.648066 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.648040 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:18:47.751559 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751524 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751711 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751576 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751711 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751604 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slcnj\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-kube-api-access-slcnj\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751711 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751642 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-config\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751711 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751665 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751739 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751773 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751795 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751810 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.751914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751851 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752123 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751921 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752123 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.751963 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752123 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752007 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752123 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752037 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752123 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752064 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752123 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752081 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752372 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752169 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752372 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752197 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752372 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752214 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.752372 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.752241 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.852919 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.852893 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.852935 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.852969 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.852999 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.853026 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.853047 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.853069 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.853111 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.853138 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.853378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.853161 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854108 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854083 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854207 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854145 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854207 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854184 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854312 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854211 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854312 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854223 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854312 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854249 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slcnj\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-kube-api-access-slcnj\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854312 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854293 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-config\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854501 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854320 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854501 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854367 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854501 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854396 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854501 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854426 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854501 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854449 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.854869 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.854845 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.857153 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.856164 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.857153 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.856511 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.857153 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.856910 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.857153 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.857101 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.857646 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.857599 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.857646 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.857618 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.858586 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.858342 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.858586 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.858551 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.858729 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.858584 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.858729 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.858705 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.859547 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.859224 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-config\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.859547 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.859396 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.859547 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.859518 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.860496 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.860472 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.860597 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.860500 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.860725 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.860706 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.866503 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.866485 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slcnj\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-kube-api-access-slcnj\") pod \"prometheus-k8s-0\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:47.951421 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:47.951365 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:18:48.607836 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:48.607798 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bcb58489-bjzm9"] Jul 28 17:18:50.407552 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:50.407517 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b94e0cb_3eb3_4c81_aeba_0117020c9de0.slice/crio-765b3df2a91b9b3165dbd7afaa07190acbea50d3c41891db0fe8704216742cd1 WatchSource:0}: Error finding container 765b3df2a91b9b3165dbd7afaa07190acbea50d3c41891db0fe8704216742cd1: Status 404 returned error can't find the container with id 765b3df2a91b9b3165dbd7afaa07190acbea50d3c41891db0fe8704216742cd1 Jul 28 17:18:50.622627 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.622584 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:18:50.626087 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:50.626062 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06362a90_e2ed_42d6_9f52_064a2156ce1b.slice/crio-d714ce3a63555ea0588253b070a5b0bf20e62aa6cb4a6204b12f48a177dfa08f WatchSource:0}: Error finding container d714ce3a63555ea0588253b070a5b0bf20e62aa6cb4a6204b12f48a177dfa08f: Status 404 returned error can't find the container with id d714ce3a63555ea0588253b070a5b0bf20e62aa6cb4a6204b12f48a177dfa08f Jul 28 17:18:50.632815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.632779 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6897794bf-wjbvf"] Jul 28 17:18:50.636190 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:50.636170 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb8bad2_b2a9_41d6_b64c_dbf23980162e.slice/crio-5c158e65d82bc41f06d2e1371b1dd089e3c19abc6d1ad314af706d2dd7738d74 WatchSource:0}: Error finding container 5c158e65d82bc41f06d2e1371b1dd089e3c19abc6d1ad314af706d2dd7738d74: Status 404 returned error can't find the container with id 5c158e65d82bc41f06d2e1371b1dd089e3c19abc6d1ad314af706d2dd7738d74 Jul 28 17:18:50.684941 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.684910 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67wqk" event={"ID":"4b94e0cb-3eb3-4c81-aeba-0117020c9de0","Type":"ContainerStarted","Data":"765b3df2a91b9b3165dbd7afaa07190acbea50d3c41891db0fe8704216742cd1"} Jul 28 17:18:50.686921 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.686894 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rflm7" event={"ID":"c0989e1e-1e45-401f-b97b-c8ec8500e786","Type":"ContainerStarted","Data":"1976073b0c70020a330008a89c8281436741d5daa6367b857ef6a6e833ce1bbd"} Jul 28 17:18:50.688417 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.688394 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-lh9t4" event={"ID":"3a95e312-298d-4ad1-b495-8f8d98a7ccb4","Type":"ContainerStarted","Data":"79e93ad689b29bcef8f52126032a8a4288750641113932a0523bb15326b128e8"} Jul 28 17:18:50.688607 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.688588 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-lh9t4" Jul 28 17:18:50.693138 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.691233 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcb58489-bjzm9" event={"ID":"6cee8893-4223-4bf6-9653-f7bfbfab1464","Type":"ContainerStarted","Data":"ef9380c19b7745c7d1a1e6fd88248ed5284d902dae5c57a197be6628d7107eda"} Jul 28 17:18:50.695085 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.694999 2554 patch_prober.go:28] interesting pod/downloads-7c97dc4555-lh9t4 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused" start-of-body= Jul 28 17:18:50.695890 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.695839 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-lh9t4" podUID="3a95e312-298d-4ad1-b495-8f8d98a7ccb4" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused" Jul 28 17:18:50.697184 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.697155 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerStarted","Data":"d714ce3a63555ea0588253b070a5b0bf20e62aa6cb4a6204b12f48a177dfa08f"} Jul 28 17:18:50.698711 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.698682 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6897794bf-wjbvf" event={"ID":"4eb8bad2-b2a9-41d6-b64c-dbf23980162e","Type":"ContainerStarted","Data":"5c158e65d82bc41f06d2e1371b1dd089e3c19abc6d1ad314af706d2dd7738d74"} Jul 28 17:18:50.709051 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.709001 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-lh9t4" podStartSLOduration=0.909377186 podStartE2EDuration="21.708985515s" totalImagesPullingTime="20.799608329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:50.708253814 +0000 UTC m=+88.926295276" watchObservedRunningTime="2026-07-28 17:18:50.708985515 +0000 UTC m=+88.927026975" Jul 28 17:18:50.730247 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.730210 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5bcb58489-bjzm9" podStartSLOduration=3.332046889 podStartE2EDuration="16.730197352s" totalImagesPullingTime="13.398150463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:50.72904106 +0000 UTC m=+88.947082525" watchObservedRunningTime="2026-07-28 17:18:50.730197352 +0000 UTC m=+88.948238814" Jul 28 17:18:50.854441 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.854415 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt"] Jul 28 17:18:50.856162 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:50.856135 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432e2472_c9a3_479e_b840_f7202a4483cc.slice/crio-73cf3de0c0c0b62b7523e1f97e9522c66e5349d346262c52f354a77b8c6a842f WatchSource:0}: Error finding container 73cf3de0c0c0b62b7523e1f97e9522c66e5349d346262c52f354a77b8c6a842f: Status 404 returned error can't find the container with id 73cf3de0c0c0b62b7523e1f97e9522c66e5349d346262c52f354a77b8c6a842f Jul 28 17:18:50.860595 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.860573 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55557df95f-8vrp5"] Jul 28 17:18:50.863855 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:50.863807 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:18:50.866722 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:18:50.866620 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8964d6df_08c7_42ce_9f23_72f47a33d993.slice/crio-ad1f8f894b1d712daff873a9548cf657081f2fc0edf4bc752cba4f8be826b293 WatchSource:0}: Error finding container ad1f8f894b1d712daff873a9548cf657081f2fc0edf4bc752cba4f8be826b293: Status 404 returned error can't find the container with id ad1f8f894b1d712daff873a9548cf657081f2fc0edf4bc752cba4f8be826b293 Jul 28 17:18:51.291594 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.291559 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:51.351510 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.351470 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:51.711854 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.711720 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" event={"ID":"432e2472-c9a3-479e-b840-f7202a4483cc","Type":"ContainerStarted","Data":"73cf3de0c0c0b62b7523e1f97e9522c66e5349d346262c52f354a77b8c6a842f"} Jul 28 17:18:51.714042 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.714010 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" event={"ID":"68b16455-4ea2-4be1-bfef-56a619605594","Type":"ContainerStarted","Data":"e78ae25487e4cb1fedd99fcebd4ddd3ee5df8d1bde2b15ebc78736b90cb1e265"} Jul 28 17:18:51.720948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.720683 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6897794bf-wjbvf" event={"ID":"4eb8bad2-b2a9-41d6-b64c-dbf23980162e","Type":"ContainerStarted","Data":"7c37c26b63dfcf1265dbbc717caf4a147c3dc33d785c5949b606a2c2af657606"} Jul 28 17:18:51.727249 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.727191 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerStarted","Data":"ad1f8f894b1d712daff873a9548cf657081f2fc0edf4bc752cba4f8be826b293"} Jul 28 17:18:51.739445 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.739424 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-lh9t4" Jul 28 17:18:51.754547 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.754498 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6897794bf-wjbvf" podStartSLOduration=9.754478179 podStartE2EDuration="9.754478179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:51.752651477 +0000 UTC m=+89.970692942" watchObservedRunningTime="2026-07-28 17:18:51.754478179 +0000 UTC m=+89.972519645" Jul 28 17:18:51.787810 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:51.787737 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:52.154352 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:52.154315 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6897794bf-wjbvf"] Jul 28 17:18:52.933157 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:52.933120 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:52.933665 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:52.933237 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:52.939824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:52.939796 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:52.966328 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:52.966287 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:53.040850 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:53.040819 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:53.105427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:53.105388 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:53.743573 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:53.743539 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:18:54.485523 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:54.485443 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:18:54.654994 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:54.654916 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:54.752222 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:54.752147 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:55.200766 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:55.200731 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:55.261176 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:55.261131 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:57.193117 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.193068 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:57.263849 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.263807 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:57.774349 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.774316 2554 generic.go:356] "Generic (PLEG): container finished" podID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" exitCode=0 Jul 28 17:18:57.774467 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.774400 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624"} Jul 28 17:18:57.778195 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.777438 2554 generic.go:356] "Generic (PLEG): container finished" podID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerID="1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62" exitCode=0 Jul 28 17:18:57.778195 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.777510 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62"} Jul 28 17:18:57.781043 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.781022 2554 generic.go:356] "Generic (PLEG): container finished" podID="4b94e0cb-3eb3-4c81-aeba-0117020c9de0" containerID="1753821e3efaedd62b453aa84cd7ecd63fa1e7e04f27afb2f2c8dcc2da705158" exitCode=0 Jul 28 17:18:57.785865 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.783484 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67wqk" event={"ID":"4b94e0cb-3eb3-4c81-aeba-0117020c9de0","Type":"ContainerDied","Data":"1753821e3efaedd62b453aa84cd7ecd63fa1e7e04f27afb2f2c8dcc2da705158"} Jul 28 17:18:57.786863 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.786836 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" event={"ID":"432e2472-c9a3-479e-b840-f7202a4483cc","Type":"ContainerStarted","Data":"cc73c5134d2154b4ecac633ae40f979e0f07d4a1f809c79aaa52d0ece5915df5"} Jul 28 17:18:57.789396 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.789188 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rflm7" event={"ID":"c0989e1e-1e45-401f-b97b-c8ec8500e786","Type":"ContainerStarted","Data":"a224ae350225e395e2789013ae13776a625d3bc6c4e98d29cb814839318c31d5"} Jul 28 17:18:57.791261 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.791232 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" event={"ID":"68b16455-4ea2-4be1-bfef-56a619605594","Type":"ContainerStarted","Data":"38f35466ce13658b1fb913cb0ecab769480720aee361e5a439ec87ca038d3aeb"} Jul 28 17:18:57.791359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.791298 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" event={"ID":"68b16455-4ea2-4be1-bfef-56a619605594","Type":"ContainerStarted","Data":"a7e346e9da0eab32f204fd9faae88ca3b5bac057f2888226c44bc5d67cbf191a"} Jul 28 17:18:57.888234 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.888189 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" podStartSLOduration=6.472104933 podStartE2EDuration="12.888168328s" totalImagesPullingTime="6.416063395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:57.83671233 +0000 UTC m=+96.054753794" watchObservedRunningTime="2026-07-28 17:18:57.888168328 +0000 UTC m=+96.106209794" Jul 28 17:18:57.932085 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:57.930680 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rflm7" podStartSLOduration=32.93806069 podStartE2EDuration="1m0.930661157s" totalImagesPullingTime="27.992600467s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:17:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:57.929488886 +0000 UTC m=+96.147530350" watchObservedRunningTime="2026-07-28 17:18:57.930661157 +0000 UTC m=+96.148702621" Jul 28 17:18:58.801726 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:58.801597 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67wqk" event={"ID":"4b94e0cb-3eb3-4c81-aeba-0117020c9de0","Type":"ContainerStarted","Data":"e62a5d75b9fdeb45cad6dec25ebe7d7128ba1cedf1b7245a0bc1abec1bbd09c5"} Jul 28 17:18:58.801726 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:58.801665 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-67wqk" event={"ID":"4b94e0cb-3eb3-4c81-aeba-0117020c9de0","Type":"ContainerStarted","Data":"c6dde7eeaa0fb3e1599863c086ab227de6c86319e318a086a605aacb62089c22"} Jul 28 17:18:58.810835 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:58.810807 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" event={"ID":"68b16455-4ea2-4be1-bfef-56a619605594","Type":"ContainerStarted","Data":"555f473c71d6ba6f682bf91d9da39c35ac19895e70ffd28d344674f658aabe23"} Jul 28 17:18:58.823676 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:58.823628 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-67wqk" podStartSLOduration=11.024454889 podStartE2EDuration="17.823608374s" totalImagesPullingTime="6.68549217s" totalInitContainerRuntime="113.661315ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:18:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:58.821700335 +0000 UTC m=+97.039741818" watchObservedRunningTime="2026-07-28 17:18:58.823608374 +0000 UTC m=+97.041649840" Jul 28 17:18:59.528423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:59.528386 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:18:59.580905 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:59.580857 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fl4xc" Jul 28 17:18:59.593185 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:18:59.593150 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:00.015817 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:00.015790 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:00.116596 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:00.116572 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:00.384086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:00.384052 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:00.821999 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:00.821962 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" event={"ID":"68b16455-4ea2-4be1-bfef-56a619605594","Type":"ContainerStarted","Data":"6e9edb74e1feb5e364e2d220b1aa974682a071bd97546f5d9dabf5e3bb5393e4"} Jul 28 17:19:01.827215 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:01.827190 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerStarted","Data":"adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be"} Jul 28 17:19:01.830333 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:01.830302 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" event={"ID":"68b16455-4ea2-4be1-bfef-56a619605594","Type":"ContainerStarted","Data":"a3b028fe7c7c3ae6398e427fe4873a65cab09fe0cc86ad68183dd7918b8f0a95"} Jul 28 17:19:01.830429 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:01.830357 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" event={"ID":"68b16455-4ea2-4be1-bfef-56a619605594","Type":"ContainerStarted","Data":"8bc7d44e8c4622a5ca6da81bb21dc989a479f6aabeaa0177044309cf0d903266"} Jul 28 17:19:01.830497 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:01.830439 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:19:01.857115 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:01.857073 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" podStartSLOduration=9.202318264 podStartE2EDuration="17.85705624s" totalImagesPullingTime="8.654737976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:01.855177579 +0000 UTC m=+100.073219054" watchObservedRunningTime="2026-07-28 17:19:01.85705624 +0000 UTC m=+100.075097702" Jul 28 17:19:02.726629 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.726536 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:02.838730 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.838701 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerStarted","Data":"e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0"} Jul 28 17:19:02.839145 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.838782 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerStarted","Data":"65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce"} Jul 28 17:19:02.839145 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.838831 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerStarted","Data":"16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8"} Jul 28 17:19:02.839145 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.838886 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerStarted","Data":"8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6"} Jul 28 17:19:02.839145 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.838925 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerStarted","Data":"626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de"} Jul 28 17:19:02.842234 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.842206 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerStarted","Data":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} Jul 28 17:19:02.842350 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.842277 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerStarted","Data":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} Jul 28 17:19:02.842350 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.842321 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerStarted","Data":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} Jul 28 17:19:02.842433 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.842361 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerStarted","Data":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} Jul 28 17:19:02.842433 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.842400 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerStarted","Data":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} Jul 28 17:19:02.842490 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.842439 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerStarted","Data":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} Jul 28 17:19:02.848674 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.848656 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-55557df95f-8vrp5" Jul 28 17:19:02.866317 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.866288 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:02.868689 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.868646 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=10.575661821 podStartE2EDuration="20.86861338s" totalImagesPullingTime="10.26746142s" totalInitContainerRuntime="25.490139ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:02.866126366 +0000 UTC m=+101.084167830" watchObservedRunningTime="2026-07-28 17:19:02.86861338 +0000 UTC m=+101.086654843" Jul 28 17:19:02.898402 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.898356 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.035202625 podStartE2EDuration="15.898338787s" totalImagesPullingTime="10.831484214s" totalInitContainerRuntime="31.651948ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:19:02.896583098 +0000 UTC m=+101.114624587" watchObservedRunningTime="2026-07-28 17:19:02.898338787 +0000 UTC m=+101.116380250" Jul 28 17:19:02.951639 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.951618 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:19:02.965040 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:02.965019 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:03.024403 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:03.024379 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:06.000701 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:06.000663 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:19:06.001212 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:06.000727 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:19:07.651039 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:07.650985 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:07.748774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:07.748741 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:07.806577 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:07.806549 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:14.529293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:14.529259 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:14.630183 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:14.630152 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:14.812946 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:14.812857 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:14.877438 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:14.877371 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:15.714398 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:15.714338 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5bcb58489-bjzm9" podUID="6cee8893-4223-4bf6-9653-f7bfbfab1464" containerName="console" containerID="cri-o://ef9380c19b7745c7d1a1e6fd88248ed5284d902dae5c57a197be6628d7107eda" gracePeriod=15 Jul 28 17:19:15.896433 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:15.896407 2554 generic.go:356] "Generic (PLEG): container finished" podID="ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3" containerID="9e74ce69b6ee14ac0c5f5f3c650a8b3eeed46da3f0751689b789ed2cbc8bc2d7" exitCode=0 Jul 28 17:19:15.896538 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:15.896478 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" event={"ID":"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3","Type":"ContainerDied","Data":"9e74ce69b6ee14ac0c5f5f3c650a8b3eeed46da3f0751689b789ed2cbc8bc2d7"} Jul 28 17:19:15.896865 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:15.896847 2554 scope.go:117] "RemoveContainer" containerID="9e74ce69b6ee14ac0c5f5f3c650a8b3eeed46da3f0751689b789ed2cbc8bc2d7" Jul 28 17:19:15.898247 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:15.898227 2554 generic.go:356] "Generic (PLEG): container finished" podID="6cee8893-4223-4bf6-9653-f7bfbfab1464" containerID="ef9380c19b7745c7d1a1e6fd88248ed5284d902dae5c57a197be6628d7107eda" exitCode=0 Jul 28 17:19:15.898330 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:15.898257 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcb58489-bjzm9" event={"ID":"6cee8893-4223-4bf6-9653-f7bfbfab1464","Type":"ContainerDied","Data":"ef9380c19b7745c7d1a1e6fd88248ed5284d902dae5c57a197be6628d7107eda"} Jul 28 17:19:15.992900 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:15.992865 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:19:16.095113 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095084 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-config\") pod \"6cee8893-4223-4bf6-9653-f7bfbfab1464\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " Jul 28 17:19:16.095264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095134 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-oauth-serving-cert\") pod \"6cee8893-4223-4bf6-9653-f7bfbfab1464\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " Jul 28 17:19:16.095264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095182 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-service-ca\") pod \"6cee8893-4223-4bf6-9653-f7bfbfab1464\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " Jul 28 17:19:16.095264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095215 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-oauth-config\") pod \"6cee8893-4223-4bf6-9653-f7bfbfab1464\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " Jul 28 17:19:16.095264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095246 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-serving-cert\") pod \"6cee8893-4223-4bf6-9653-f7bfbfab1464\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " Jul 28 17:19:16.095475 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095286 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9nw4b\" (UniqueName: \"kubernetes.io/projected/6cee8893-4223-4bf6-9653-f7bfbfab1464-kube-api-access-9nw4b\") pod \"6cee8893-4223-4bf6-9653-f7bfbfab1464\" (UID: \"6cee8893-4223-4bf6-9653-f7bfbfab1464\") " Jul 28 17:19:16.095475 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095423 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-config" (OuterVolumeSpecName: "console-config") pod "6cee8893-4223-4bf6-9653-f7bfbfab1464" (UID: "6cee8893-4223-4bf6-9653-f7bfbfab1464"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:16.095573 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095553 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-service-ca" (OuterVolumeSpecName: "service-ca") pod "6cee8893-4223-4bf6-9653-f7bfbfab1464" (UID: "6cee8893-4223-4bf6-9653-f7bfbfab1464"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:16.095618 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095565 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6cee8893-4223-4bf6-9653-f7bfbfab1464" (UID: "6cee8893-4223-4bf6-9653-f7bfbfab1464"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:16.095618 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.095606 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:16.097435 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.097406 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6cee8893-4223-4bf6-9653-f7bfbfab1464" (UID: "6cee8893-4223-4bf6-9653-f7bfbfab1464"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:16.097559 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.097531 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6cee8893-4223-4bf6-9653-f7bfbfab1464" (UID: "6cee8893-4223-4bf6-9653-f7bfbfab1464"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:16.097559 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.097548 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cee8893-4223-4bf6-9653-f7bfbfab1464-kube-api-access-9nw4b" (OuterVolumeSpecName: "kube-api-access-9nw4b") pod "6cee8893-4223-4bf6-9653-f7bfbfab1464" (UID: "6cee8893-4223-4bf6-9653-f7bfbfab1464"). InnerVolumeSpecName "kube-api-access-9nw4b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:19:16.196536 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.196512 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-oauth-serving-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:16.196536 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.196533 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6cee8893-4223-4bf6-9653-f7bfbfab1464-service-ca\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:16.196693 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.196542 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-oauth-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:16.196693 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.196550 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6cee8893-4223-4bf6-9653-f7bfbfab1464-console-serving-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:16.196693 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.196559 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9nw4b\" (UniqueName: \"kubernetes.io/projected/6cee8893-4223-4bf6-9653-f7bfbfab1464-kube-api-access-9nw4b\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:16.904177 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.904154 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5bcb58489-bjzm9" Jul 28 17:19:16.904613 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.904206 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5bcb58489-bjzm9" event={"ID":"6cee8893-4223-4bf6-9653-f7bfbfab1464","Type":"ContainerDied","Data":"f8180615f26d6641e94a43423d7beebfb56747f680658a490c661e65685b3376"} Jul 28 17:19:16.904613 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.904274 2554 scope.go:117] "RemoveContainer" containerID="ef9380c19b7745c7d1a1e6fd88248ed5284d902dae5c57a197be6628d7107eda" Jul 28 17:19:16.906607 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.906560 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc" event={"ID":"ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3","Type":"ContainerStarted","Data":"cf2f4eb39d4261b870f82ff522844afd5f88178475c4b1e561f31001d798b650"} Jul 28 17:19:16.923734 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.923706 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5bcb58489-bjzm9"] Jul 28 17:19:16.934359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:16.934334 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5bcb58489-bjzm9"] Jul 28 17:19:18.336301 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:18.336271 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cee8893-4223-4bf6-9653-f7bfbfab1464" path="/var/lib/kubelet/pods/6cee8893-4223-4bf6-9653-f7bfbfab1464/volumes" Jul 28 17:19:18.763903 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:18.763859 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6897794bf-wjbvf" podUID="4eb8bad2-b2a9-41d6-b64c-dbf23980162e" containerName="console" containerID="cri-o://7c37c26b63dfcf1265dbbc717caf4a147c3dc33d785c5949b606a2c2af657606" gracePeriod=15 Jul 28 17:19:18.916899 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:18.916857 2554 generic.go:356] "Generic (PLEG): container finished" podID="4eb8bad2-b2a9-41d6-b64c-dbf23980162e" containerID="7c37c26b63dfcf1265dbbc717caf4a147c3dc33d785c5949b606a2c2af657606" exitCode=0 Jul 28 17:19:18.916996 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:18.916909 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6897794bf-wjbvf" event={"ID":"4eb8bad2-b2a9-41d6-b64c-dbf23980162e","Type":"ContainerDied","Data":"7c37c26b63dfcf1265dbbc717caf4a147c3dc33d785c5949b606a2c2af657606"} Jul 28 17:19:19.063696 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.063665 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:19:19.117678 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.117652 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cjfs\" (UniqueName: \"kubernetes.io/projected/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-kube-api-access-7cjfs\") pod \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " Jul 28 17:19:19.117802 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.117706 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-config\") pod \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " Jul 28 17:19:19.117802 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.117727 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-trusted-ca-bundle\") pod \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " Jul 28 17:19:19.117802 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.117763 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-oauth-config\") pod \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " Jul 28 17:19:19.117802 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.117788 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-serving-cert\") pod \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " Jul 28 17:19:19.118004 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.117813 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-service-ca\") pod \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " Jul 28 17:19:19.118004 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.117839 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-oauth-serving-cert\") pod \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\" (UID: \"4eb8bad2-b2a9-41d6-b64c-dbf23980162e\") " Jul 28 17:19:19.118149 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.118116 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-config" (OuterVolumeSpecName: "console-config") pod "4eb8bad2-b2a9-41d6-b64c-dbf23980162e" (UID: "4eb8bad2-b2a9-41d6-b64c-dbf23980162e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:19.118300 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.118276 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4eb8bad2-b2a9-41d6-b64c-dbf23980162e" (UID: "4eb8bad2-b2a9-41d6-b64c-dbf23980162e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:19.118375 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.118280 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-service-ca" (OuterVolumeSpecName: "service-ca") pod "4eb8bad2-b2a9-41d6-b64c-dbf23980162e" (UID: "4eb8bad2-b2a9-41d6-b64c-dbf23980162e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:19.118427 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.118386 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4eb8bad2-b2a9-41d6-b64c-dbf23980162e" (UID: "4eb8bad2-b2a9-41d6-b64c-dbf23980162e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:19.119673 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.119652 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-kube-api-access-7cjfs" (OuterVolumeSpecName: "kube-api-access-7cjfs") pod "4eb8bad2-b2a9-41d6-b64c-dbf23980162e" (UID: "4eb8bad2-b2a9-41d6-b64c-dbf23980162e"). InnerVolumeSpecName "kube-api-access-7cjfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:19:19.119775 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.119683 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4eb8bad2-b2a9-41d6-b64c-dbf23980162e" (UID: "4eb8bad2-b2a9-41d6-b64c-dbf23980162e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:19.119775 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.119698 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4eb8bad2-b2a9-41d6-b64c-dbf23980162e" (UID: "4eb8bad2-b2a9-41d6-b64c-dbf23980162e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:19.218790 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.218764 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cjfs\" (UniqueName: \"kubernetes.io/projected/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-kube-api-access-7cjfs\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:19.218790 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.218789 2554 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:19.218948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.218803 2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-trusted-ca-bundle\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:19.218948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.218814 2554 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-oauth-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:19.218948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.218826 2554 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-console-serving-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:19.218948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.218839 2554 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-service-ca\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:19.218948 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.218852 2554 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4eb8bad2-b2a9-41d6-b64c-dbf23980162e-oauth-serving-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:19:19.926296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.926269 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6897794bf-wjbvf" Jul 28 17:19:19.926664 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.926264 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6897794bf-wjbvf" event={"ID":"4eb8bad2-b2a9-41d6-b64c-dbf23980162e","Type":"ContainerDied","Data":"5c158e65d82bc41f06d2e1371b1dd089e3c19abc6d1ad314af706d2dd7738d74"} Jul 28 17:19:19.926664 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.926421 2554 scope.go:117] "RemoveContainer" containerID="7c37c26b63dfcf1265dbbc717caf4a147c3dc33d785c5949b606a2c2af657606" Jul 28 17:19:19.948172 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.948151 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6897794bf-wjbvf"] Jul 28 17:19:19.951845 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:19.951826 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6897794bf-wjbvf"] Jul 28 17:19:20.336766 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:20.336744 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4eb8bad2-b2a9-41d6-b64c-dbf23980162e" path="/var/lib/kubelet/pods/4eb8bad2-b2a9-41d6-b64c-dbf23980162e/volumes" Jul 28 17:19:21.936259 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:21.936233 2554 generic.go:356] "Generic (PLEG): container finished" podID="08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced" containerID="cccdbce2fc2573db484100881e6ba619da9031ebba01bda08fbb14378b27e83e" exitCode=0 Jul 28 17:19:21.936609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:21.936304 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" event={"ID":"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced","Type":"ContainerDied","Data":"cccdbce2fc2573db484100881e6ba619da9031ebba01bda08fbb14378b27e83e"} Jul 28 17:19:21.936609 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:21.936598 2554 scope.go:117] "RemoveContainer" containerID="cccdbce2fc2573db484100881e6ba619da9031ebba01bda08fbb14378b27e83e" Jul 28 17:19:22.941335 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:22.941294 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gbhfm" event={"ID":"08e7b8a7-10a8-44c8-9e4d-e5f2bdc2aced","Type":"ContainerStarted","Data":"c1debd98a441bd072e9c401a30f1262d18f5266cd16b0afb22f05d02afa15292"} Jul 28 17:19:26.005704 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:26.005674 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:19:26.009234 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:26.009211 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5d5b5d9d67-5ddtt" Jul 28 17:19:38.095166 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:38.095122 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:47.951961 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:47.951915 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:19:47.973280 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:47.973251 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:19:48.058114 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:48.058092 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:19:49.805312 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:49.805268 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:57.528832 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:57.528780 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:19:57.629897 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:19:57.629837 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:01.840236 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:01.840191 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:20:01.840736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:01.840582 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="alertmanager" containerID="cri-o://adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be" gracePeriod=120 Jul 28 17:20:01.840736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:01.840635 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-metric" containerID="cri-o://65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce" gracePeriod=120 Jul 28 17:20:01.840736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:01.840655 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-web" containerID="cri-o://8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6" gracePeriod=120 Jul 28 17:20:01.840736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:01.840711 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="prom-label-proxy" containerID="cri-o://e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0" gracePeriod=120 Jul 28 17:20:01.840736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:01.840703 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy" containerID="cri-o://16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8" gracePeriod=120 Jul 28 17:20:01.841040 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:01.840665 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="config-reloader" containerID="cri-o://626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de" gracePeriod=120 Jul 28 17:20:02.102480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102425 2554 generic.go:356] "Generic (PLEG): container finished" podID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerID="e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0" exitCode=0 Jul 28 17:20:02.102480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102446 2554 generic.go:356] "Generic (PLEG): container finished" podID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerID="65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce" exitCode=0 Jul 28 17:20:02.102480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102451 2554 generic.go:356] "Generic (PLEG): container finished" podID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerID="16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8" exitCode=0 Jul 28 17:20:02.102480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102457 2554 generic.go:356] "Generic (PLEG): container finished" podID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerID="626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de" exitCode=0 Jul 28 17:20:02.102480 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102462 2554 generic.go:356] "Generic (PLEG): container finished" podID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerID="adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be" exitCode=0 Jul 28 17:20:02.102688 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102492 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0"} Jul 28 17:20:02.102688 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102563 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce"} Jul 28 17:20:02.102688 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102588 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8"} Jul 28 17:20:02.102688 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102611 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de"} Jul 28 17:20:02.102688 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:02.102633 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be"} Jul 28 17:20:03.084465 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.084441 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.108785 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.108764 2554 generic.go:356] "Generic (PLEG): container finished" podID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerID="8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6" exitCode=0 Jul 28 17:20:03.108894 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.108815 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6"} Jul 28 17:20:03.108894 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.108828 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.108894 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.108866 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8964d6df-08c7-42ce-9f23-72f47a33d993","Type":"ContainerDied","Data":"ad1f8f894b1d712daff873a9548cf657081f2fc0edf4bc752cba4f8be826b293"} Jul 28 17:20:03.109037 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.108909 2554 scope.go:117] "RemoveContainer" containerID="e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0" Jul 28 17:20:03.115852 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.115810 2554 scope.go:117] "RemoveContainer" containerID="65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce" Jul 28 17:20:03.124423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.124405 2554 scope.go:117] "RemoveContainer" containerID="16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8" Jul 28 17:20:03.132124 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.132107 2554 scope.go:117] "RemoveContainer" containerID="8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6" Jul 28 17:20:03.139129 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.139112 2554 scope.go:117] "RemoveContainer" containerID="626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de" Jul 28 17:20:03.147557 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.147534 2554 scope.go:117] "RemoveContainer" containerID="adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be" Jul 28 17:20:03.154265 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.154248 2554 scope.go:117] "RemoveContainer" containerID="1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62" Jul 28 17:20:03.161744 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.161724 2554 scope.go:117] "RemoveContainer" containerID="e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0" Jul 28 17:20:03.162149 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:03.162126 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0\": container with ID starting with e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0 not found: ID does not exist" containerID="e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0" Jul 28 17:20:03.162220 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162155 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0"} err="failed to get container status \"e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0\": rpc error: code = NotFound desc = could not find container \"e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0\": container with ID starting with e88c03bb2a55136f15ca7513e590d4cba0d867731c30cb28ac24c89d2316b1a0 not found: ID does not exist" Jul 28 17:20:03.162220 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162185 2554 scope.go:117] "RemoveContainer" containerID="65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce" Jul 28 17:20:03.162414 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:03.162395 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce\": container with ID starting with 65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce not found: ID does not exist" containerID="65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce" Jul 28 17:20:03.162468 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162420 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce"} err="failed to get container status \"65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce\": rpc error: code = NotFound desc = could not find container \"65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce\": container with ID starting with 65748a946d537789ee4bae3d73fce28f03dd387bc519aa9c2e4cf76df398ecce not found: ID does not exist" Jul 28 17:20:03.162468 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162436 2554 scope.go:117] "RemoveContainer" containerID="16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8" Jul 28 17:20:03.162697 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:03.162681 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8\": container with ID starting with 16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8 not found: ID does not exist" containerID="16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8" Jul 28 17:20:03.162751 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162700 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8"} err="failed to get container status \"16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8\": rpc error: code = NotFound desc = could not find container \"16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8\": container with ID starting with 16c6168af19137cf626db8874fd370734f296f8085866e1d67e6987fd4b3cce8 not found: ID does not exist" Jul 28 17:20:03.162751 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162714 2554 scope.go:117] "RemoveContainer" containerID="8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6" Jul 28 17:20:03.162949 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:03.162933 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6\": container with ID starting with 8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6 not found: ID does not exist" containerID="8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6" Jul 28 17:20:03.163012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162953 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6"} err="failed to get container status \"8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6\": rpc error: code = NotFound desc = could not find container \"8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6\": container with ID starting with 8822566e1750041c92c3d14f4e57fdb776924c0d8761ec43ebbfe852dc4930a6 not found: ID does not exist" Jul 28 17:20:03.163012 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.162970 2554 scope.go:117] "RemoveContainer" containerID="626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de" Jul 28 17:20:03.163212 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:03.163195 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de\": container with ID starting with 626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de not found: ID does not exist" containerID="626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de" Jul 28 17:20:03.163262 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.163223 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de"} err="failed to get container status \"626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de\": rpc error: code = NotFound desc = could not find container \"626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de\": container with ID starting with 626c08eb05d9c678cd6b7a0b249115b1298a5861711138daf53ebc1592f127de not found: ID does not exist" Jul 28 17:20:03.163262 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.163237 2554 scope.go:117] "RemoveContainer" containerID="adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be" Jul 28 17:20:03.163453 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:03.163438 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be\": container with ID starting with adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be not found: ID does not exist" containerID="adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be" Jul 28 17:20:03.163506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.163457 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be"} err="failed to get container status \"adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be\": rpc error: code = NotFound desc = could not find container \"adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be\": container with ID starting with adafdf0617ead2f01ebe9f98c0d0fab46e1b68861b61a33cd8dc1a34519705be not found: ID does not exist" Jul 28 17:20:03.163506 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.163470 2554 scope.go:117] "RemoveContainer" containerID="1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62" Jul 28 17:20:03.163683 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:03.163666 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62\": container with ID starting with 1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62 not found: ID does not exist" containerID="1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62" Jul 28 17:20:03.163752 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.163686 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62"} err="failed to get container status \"1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62\": rpc error: code = NotFound desc = could not find container \"1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62\": container with ID starting with 1577a7f70614e33944cb0644fb09d1e38e65c465693d2ff5692e60efe1b2dc62 not found: ID does not exist" Jul 28 17:20:03.239984 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.239963 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-main-db\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240070 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.239995 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240070 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240018 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-config-volume\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240070 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240038 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-cluster-tls-config\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240070 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240060 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240085 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-main-tls\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240104 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-config-out\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240125 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-web\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240147 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-tls-assets\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240193 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-web-config\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240217 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-metrics-client-ca\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240255 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240240 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4kwh\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-kube-api-access-b4kwh\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.240585 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240285 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:20:03.240779 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.240691 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-trusted-ca-bundle\") pod \"8964d6df-08c7-42ce-9f23-72f47a33d993\" (UID: \"8964d6df-08c7-42ce-9f23-72f47a33d993\") " Jul 28 17:20:03.241547 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.241522 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:03.241800 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.241773 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:03.242844 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.242814 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-config-volume" (OuterVolumeSpecName: "config-volume") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:03.243558 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.243533 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-config-out" (OuterVolumeSpecName: "config-out") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:20:03.243662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.243570 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:03.243662 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.243628 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:03.243902 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.243857 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:03.244002 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.243949 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:03.244448 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.244423 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:20:03.244695 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.244676 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-kube-api-access-b4kwh" (OuterVolumeSpecName: "kube-api-access-b4kwh") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "kube-api-access-b4kwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:20:03.247331 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.247311 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:03.253513 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.253493 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-web-config" (OuterVolumeSpecName: "web-config") pod "8964d6df-08c7-42ce-9f23-72f47a33d993" (UID: "8964d6df-08c7-42ce-9f23-72f47a33d993"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:03.341494 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341474 2554 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-tls-assets\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341509 2554 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-web-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341519 2554 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-metrics-client-ca\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341527 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4kwh\" (UniqueName: \"kubernetes.io/projected/8964d6df-08c7-42ce-9f23-72f47a33d993-kube-api-access-b4kwh\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341537 2554 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341545 2554 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-alertmanager-main-db\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341553 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341561 2554 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-config-volume\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341760 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341569 2554 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-cluster-tls-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341760 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341577 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341760 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341586 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-main-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341760 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341594 2554 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8964d6df-08c7-42ce-9f23-72f47a33d993-config-out\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.341760 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.341602 2554 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8964d6df-08c7-42ce-9f23-72f47a33d993-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:03.434083 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.434055 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:20:03.439533 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.439512 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:20:03.465943 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.465917 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:20:03.466216 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466201 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="prom-label-proxy" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466217 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="prom-label-proxy" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466227 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466232 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466239 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="alertmanager" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466246 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="alertmanager" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466253 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cee8893-4223-4bf6-9653-f7bfbfab1464" containerName="console" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466258 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cee8893-4223-4bf6-9653-f7bfbfab1464" containerName="console" Jul 28 17:20:03.466263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466264 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-metric" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466269 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-metric" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466291 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-web" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466297 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-web" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466306 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="config-reloader" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466311 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="config-reloader" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466318 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="init-config-reloader" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466324 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="init-config-reloader" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466333 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4eb8bad2-b2a9-41d6-b64c-dbf23980162e" containerName="console" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466339 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4eb8bad2-b2a9-41d6-b64c-dbf23980162e" containerName="console" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466382 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="alertmanager" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466390 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="config-reloader" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466395 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4eb8bad2-b2a9-41d6-b64c-dbf23980162e" containerName="console" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466400 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cee8893-4223-4bf6-9653-f7bfbfab1464" containerName="console" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466405 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="prom-label-proxy" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466410 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466415 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-web" Jul 28 17:20:03.466486 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.466419 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" containerName="kube-rbac-proxy-metric" Jul 28 17:20:03.470213 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.470198 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.473067 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473045 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 17:20:03.473183 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473164 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qw8dm\"" Jul 28 17:20:03.473243 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473169 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 17:20:03.473243 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473169 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 17:20:03.473364 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473284 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 17:20:03.473428 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473410 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 17:20:03.473810 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473794 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 17:20:03.473913 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473796 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 17:20:03.473966 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.473799 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 17:20:03.478995 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.478976 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 17:20:03.483454 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.483436 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:20:03.644612 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644588 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644618 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644632 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-web-config\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644649 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644666 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644860 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644741 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644860 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644799 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-config-out\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644860 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644843 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644988 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644865 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644988 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644933 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.644988 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644958 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.645085 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.644994 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.645085 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.645010 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2nlg\" (UniqueName: \"kubernetes.io/projected/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-kube-api-access-w2nlg\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745391 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745423 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745446 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745601 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745579 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745636 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745623 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745682 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745646 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2nlg\" (UniqueName: \"kubernetes.io/projected/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-kube-api-access-w2nlg\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745732 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745679 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745732 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745711 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745738 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-web-config\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745761 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745786 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.745824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745817 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.746054 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.745844 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-config-out\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.746357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.746324 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.748370 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.748340 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749047 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.748508 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749047 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.748660 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749047 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.748959 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749047 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.748968 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749047 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.748976 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-config-volume\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749358 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.749225 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-config-out\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749358 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.749331 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.749676 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.749648 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.750190 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.750165 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-web-config\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.750801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.750781 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.754346 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.754323 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2nlg\" (UniqueName: \"kubernetes.io/projected/004a4ee6-46fc-4fc2-b8a4-ece004bacb0a-kube-api-access-w2nlg\") pod \"alertmanager-main-0\" (UID: \"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.780620 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.780601 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 17:20:03.910323 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:03.910270 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 17:20:03.912330 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:20:03.912308 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod004a4ee6_46fc_4fc2_b8a4_ece004bacb0a.slice/crio-ef7b06c579f38a2cfe978506979df85ddbe25a16c22d762a9723fa6aaffb4a2a WatchSource:0}: Error finding container ef7b06c579f38a2cfe978506979df85ddbe25a16c22d762a9723fa6aaffb4a2a: Status 404 returned error can't find the container with id ef7b06c579f38a2cfe978506979df85ddbe25a16c22d762a9723fa6aaffb4a2a Jul 28 17:20:04.113536 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:04.113507 2554 generic.go:356] "Generic (PLEG): container finished" podID="004a4ee6-46fc-4fc2-b8a4-ece004bacb0a" containerID="d74f65410f6d83e28fceedb89e4d500c8c2ecd0b0bc7f3ba2dda8327cacc9df1" exitCode=0 Jul 28 17:20:04.113900 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:04.113591 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerDied","Data":"d74f65410f6d83e28fceedb89e4d500c8c2ecd0b0bc7f3ba2dda8327cacc9df1"} Jul 28 17:20:04.113900 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:04.113672 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerStarted","Data":"ef7b06c579f38a2cfe978506979df85ddbe25a16c22d762a9723fa6aaffb4a2a"} Jul 28 17:20:04.338584 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:04.338565 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8964d6df-08c7-42ce-9f23-72f47a33d993" path="/var/lib/kubelet/pods/8964d6df-08c7-42ce-9f23-72f47a33d993/volumes" Jul 28 17:20:05.121374 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:05.121343 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerStarted","Data":"2bb076e9c288a49ab1a53198747e6fda65bcedfd5f5f51a59aa490344d9a33bc"} Jul 28 17:20:05.121710 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:05.121436 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerStarted","Data":"b96d7b1a91a2dd71029711de6e638ec328f4a6f5aa0db683721b1a9b7133c904"} Jul 28 17:20:05.121710 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:05.121464 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerStarted","Data":"13bbede8d07e185931185c54e45afbe61e16094fde871c025939a801b7737119"} Jul 28 17:20:05.121710 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:05.121489 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerStarted","Data":"28f2f25e43333f10e9afa3176d55d8590042a4761541bead8785d26a2128dee8"} Jul 28 17:20:05.121710 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:05.121510 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerStarted","Data":"eeabbe45c34fd0f1ec5a3e15cf2b6b13195217628c5bdea2addd25580a351b6e"} Jul 28 17:20:05.121710 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:05.121537 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"004a4ee6-46fc-4fc2-b8a4-ece004bacb0a","Type":"ContainerStarted","Data":"21b4b0794861321f33c8a7139da0ebf7411ad1643223db79131f4e7256bb3df8"} Jul 28 17:20:05.151515 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:05.151433 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.136235972 podStartE2EDuration="2.15141854s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.182568ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:20:05.148755133 +0000 UTC m=+163.366796595" watchObservedRunningTime="2026-07-28 17:20:05.15141854 +0000 UTC m=+163.369460001" Jul 28 17:20:06.094644 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.094603 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:20:06.094952 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.094919 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="prometheus" containerID="cri-o://2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" gracePeriod=600 Jul 28 17:20:06.095035 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.094938 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy" containerID="cri-o://fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" gracePeriod=600 Jul 28 17:20:06.095035 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.094967 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" gracePeriod=600 Jul 28 17:20:06.095140 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.094944 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="config-reloader" containerID="cri-o://ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" gracePeriod=600 Jul 28 17:20:06.095140 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.094938 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="thanos-sidecar" containerID="cri-o://795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" gracePeriod=600 Jul 28 17:20:06.095224 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.094972 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-web" containerID="cri-o://113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" gracePeriod=600 Jul 28 17:20:06.340492 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.340472 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:06.366967 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.366911 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-1\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.366967 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.366953 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-metrics-client-certs\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.366993 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-serving-certs-ca-bundle\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367023 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367046 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-tls\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367073 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-config-out\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367108 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-web-config\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367136 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-tls-assets\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367168 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-kubelet-serving-ca-bundle\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367193 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-grpc-tls\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367220 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-metrics-client-ca\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367246 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-slcnj\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-kube-api-access-slcnj\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367267 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367285 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-2\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367292 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367307 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-thanos-prometheus-http-client-file\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367343 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-kube-rbac-proxy\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367378 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-db\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367409 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-trusted-ca-bundle\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367442 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-config\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367468 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-0\") pod \"06362a90-e2ed-42d6-9f52-064a2156ce1b\" (UID: \"06362a90-e2ed-42d6-9f52-064a2156ce1b\") " Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367504 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367596 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367750 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367769 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.367815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367784 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-metrics-client-ca\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.368296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.367937 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:06.368347 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.368329 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:06.368628 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.368604 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:06.369258 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.369221 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:20:06.369404 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.369378 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.370489 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.370442 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.370628 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.370498 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.370628 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.370612 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:20:06.370826 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.370805 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.370826 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.370826 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.371397 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.371368 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.371529 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.371412 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-config" (OuterVolumeSpecName: "config") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.371529 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.371465 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:20:06.371792 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.371766 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-config-out" (OuterVolumeSpecName: "config-out") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:20:06.371893 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.371768 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-kube-api-access-slcnj" (OuterVolumeSpecName: "kube-api-access-slcnj") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "kube-api-access-slcnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:20:06.372320 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.372298 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.383436 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.383415 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-web-config" (OuterVolumeSpecName: "web-config") pod "06362a90-e2ed-42d6-9f52-064a2156ce1b" (UID: "06362a90-e2ed-42d6-9f52-064a2156ce1b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:20:06.468269 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468251 2554 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-web-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468269 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468270 2554 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-tls-assets\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468280 2554 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468288 2554 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-grpc-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468297 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-slcnj\" (UniqueName: \"kubernetes.io/projected/06362a90-e2ed-42d6-9f52-064a2156ce1b-kube-api-access-slcnj\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468306 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468315 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468323 2554 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468333 2554 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-kube-rbac-proxy\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468341 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-db\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468349 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468357 2554 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468365 2554 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/06362a90-e2ed-42d6-9f52-064a2156ce1b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468373 2554 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-metrics-client-certs\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468379 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468381 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468710 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468389 2554 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/06362a90-e2ed-42d6-9f52-064a2156ce1b-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:06.468710 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:06.468397 2554 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/06362a90-e2ed-42d6-9f52-064a2156ce1b-config-out\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:20:07.135282 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135254 2554 generic.go:356] "Generic (PLEG): container finished" podID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" exitCode=0 Jul 28 17:20:07.135282 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135276 2554 generic.go:356] "Generic (PLEG): container finished" podID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" exitCode=0 Jul 28 17:20:07.135282 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135282 2554 generic.go:356] "Generic (PLEG): container finished" podID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" exitCode=0 Jul 28 17:20:07.135282 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135288 2554 generic.go:356] "Generic (PLEG): container finished" podID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" exitCode=0 Jul 28 17:20:07.135461 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135294 2554 generic.go:356] "Generic (PLEG): container finished" podID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" exitCode=0 Jul 28 17:20:07.135461 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135299 2554 generic.go:356] "Generic (PLEG): container finished" podID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" exitCode=0 Jul 28 17:20:07.135461 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135336 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.135461 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135336 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} Jul 28 17:20:07.135461 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135411 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} Jul 28 17:20:07.135461 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135446 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} Jul 28 17:20:07.135461 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135448 2554 scope.go:117] "RemoveContainer" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.135666 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135471 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} Jul 28 17:20:07.135666 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135497 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} Jul 28 17:20:07.135666 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135526 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} Jul 28 17:20:07.135666 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.135553 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"06362a90-e2ed-42d6-9f52-064a2156ce1b","Type":"ContainerDied","Data":"d714ce3a63555ea0588253b070a5b0bf20e62aa6cb4a6204b12f48a177dfa08f"} Jul 28 17:20:07.143566 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.143549 2554 scope.go:117] "RemoveContainer" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.150756 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.150740 2554 scope.go:117] "RemoveContainer" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.157357 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.157338 2554 scope.go:117] "RemoveContainer" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.161451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.161427 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:20:07.164669 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.164654 2554 scope.go:117] "RemoveContainer" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.167952 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.167746 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:20:07.177075 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.177057 2554 scope.go:117] "RemoveContainer" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.183814 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.183796 2554 scope.go:117] "RemoveContainer" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.190192 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.190171 2554 scope.go:117] "RemoveContainer" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.190413 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:07.190394 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": container with ID starting with d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6 not found: ID does not exist" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.190483 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.190420 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} err="failed to get container status \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": rpc error: code = NotFound desc = could not find container \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": container with ID starting with d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6 not found: ID does not exist" Jul 28 17:20:07.190483 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.190435 2554 scope.go:117] "RemoveContainer" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.190660 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:07.190642 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": container with ID starting with fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496 not found: ID does not exist" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.190698 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.190665 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} err="failed to get container status \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": rpc error: code = NotFound desc = could not find container \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": container with ID starting with fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496 not found: ID does not exist" Jul 28 17:20:07.190698 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.190681 2554 scope.go:117] "RemoveContainer" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.190923 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:07.190862 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": container with ID starting with 113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140 not found: ID does not exist" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.190980 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.190945 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} err="failed to get container status \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": rpc error: code = NotFound desc = could not find container \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": container with ID starting with 113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140 not found: ID does not exist" Jul 28 17:20:07.190980 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.190959 2554 scope.go:117] "RemoveContainer" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.191200 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:07.191186 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": container with ID starting with 795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109 not found: ID does not exist" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.191244 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191204 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} err="failed to get container status \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": rpc error: code = NotFound desc = could not find container \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": container with ID starting with 795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109 not found: ID does not exist" Jul 28 17:20:07.191244 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191216 2554 scope.go:117] "RemoveContainer" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.191416 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:07.191401 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": container with ID starting with ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8 not found: ID does not exist" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.191451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191419 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} err="failed to get container status \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": rpc error: code = NotFound desc = could not find container \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": container with ID starting with ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8 not found: ID does not exist" Jul 28 17:20:07.191451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191431 2554 scope.go:117] "RemoveContainer" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.191638 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:07.191624 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": container with ID starting with 2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029 not found: ID does not exist" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.191675 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191640 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} err="failed to get container status \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": rpc error: code = NotFound desc = could not find container \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": container with ID starting with 2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029 not found: ID does not exist" Jul 28 17:20:07.191675 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191652 2554 scope.go:117] "RemoveContainer" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.191869 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:20:07.191852 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": container with ID starting with b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624 not found: ID does not exist" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.191939 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191892 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624"} err="failed to get container status \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": rpc error: code = NotFound desc = could not find container \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": container with ID starting with b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624 not found: ID does not exist" Jul 28 17:20:07.191939 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.191907 2554 scope.go:117] "RemoveContainer" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.192171 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192149 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} err="failed to get container status \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": rpc error: code = NotFound desc = could not find container \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": container with ID starting with d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6 not found: ID does not exist" Jul 28 17:20:07.192171 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192170 2554 scope.go:117] "RemoveContainer" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.192405 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192386 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} err="failed to get container status \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": rpc error: code = NotFound desc = could not find container \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": container with ID starting with fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496 not found: ID does not exist" Jul 28 17:20:07.192451 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192405 2554 scope.go:117] "RemoveContainer" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.192636 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192617 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} err="failed to get container status \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": rpc error: code = NotFound desc = could not find container \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": container with ID starting with 113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140 not found: ID does not exist" Jul 28 17:20:07.192707 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192637 2554 scope.go:117] "RemoveContainer" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.192918 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192901 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} err="failed to get container status \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": rpc error: code = NotFound desc = could not find container \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": container with ID starting with 795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109 not found: ID does not exist" Jul 28 17:20:07.192987 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.192919 2554 scope.go:117] "RemoveContainer" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.193153 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.193127 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} err="failed to get container status \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": rpc error: code = NotFound desc = could not find container \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": container with ID starting with ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8 not found: ID does not exist" Jul 28 17:20:07.193153 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.193147 2554 scope.go:117] "RemoveContainer" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.193558 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.193530 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} err="failed to get container status \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": rpc error: code = NotFound desc = could not find container \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": container with ID starting with 2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029 not found: ID does not exist" Jul 28 17:20:07.193681 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.193558 2554 scope.go:117] "RemoveContainer" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.193914 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.193886 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624"} err="failed to get container status \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": rpc error: code = NotFound desc = could not find container \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": container with ID starting with b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624 not found: ID does not exist" Jul 28 17:20:07.194088 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.193916 2554 scope.go:117] "RemoveContainer" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.194199 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194179 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} err="failed to get container status \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": rpc error: code = NotFound desc = could not find container \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": container with ID starting with d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6 not found: ID does not exist" Jul 28 17:20:07.194263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194201 2554 scope.go:117] "RemoveContainer" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.194460 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194442 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} err="failed to get container status \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": rpc error: code = NotFound desc = could not find container \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": container with ID starting with fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496 not found: ID does not exist" Jul 28 17:20:07.194529 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194460 2554 scope.go:117] "RemoveContainer" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.194656 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194639 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} err="failed to get container status \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": rpc error: code = NotFound desc = could not find container \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": container with ID starting with 113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140 not found: ID does not exist" Jul 28 17:20:07.194714 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194657 2554 scope.go:117] "RemoveContainer" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.194906 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194864 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} err="failed to get container status \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": rpc error: code = NotFound desc = could not find container \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": container with ID starting with 795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109 not found: ID does not exist" Jul 28 17:20:07.194987 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.194907 2554 scope.go:117] "RemoveContainer" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.195119 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195102 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} err="failed to get container status \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": rpc error: code = NotFound desc = could not find container \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": container with ID starting with ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8 not found: ID does not exist" Jul 28 17:20:07.195184 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195120 2554 scope.go:117] "RemoveContainer" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.195263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195244 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:20:07.195344 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195316 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} err="failed to get container status \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": rpc error: code = NotFound desc = could not find container \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": container with ID starting with 2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029 not found: ID does not exist" Jul 28 17:20:07.195383 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195345 2554 scope.go:117] "RemoveContainer" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.195554 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195535 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624"} err="failed to get container status \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": rpc error: code = NotFound desc = could not find container \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": container with ID starting with b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624 not found: ID does not exist" Jul 28 17:20:07.195554 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195548 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="prometheus" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195555 2554 scope.go:117] "RemoveContainer" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195560 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="prometheus" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195569 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195575 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195582 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="thanos-sidecar" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195587 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="thanos-sidecar" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195594 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-thanos" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195600 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-thanos" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195614 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-web" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195619 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-web" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195632 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="config-reloader" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195640 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="config-reloader" Jul 28 17:20:07.195653 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195653 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="init-config-reloader" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195660 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="init-config-reloader" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195714 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="config-reloader" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195726 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-thanos" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195737 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195745 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="prometheus" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195753 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="thanos-sidecar" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195760 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" containerName="kube-rbac-proxy-web" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195760 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} err="failed to get container status \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": rpc error: code = NotFound desc = could not find container \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": container with ID starting with d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6 not found: ID does not exist" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.195780 2554 scope.go:117] "RemoveContainer" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196025 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} err="failed to get container status \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": rpc error: code = NotFound desc = could not find container \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": container with ID starting with fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496 not found: ID does not exist" Jul 28 17:20:07.196086 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196040 2554 scope.go:117] "RemoveContainer" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.196429 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196262 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} err="failed to get container status \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": rpc error: code = NotFound desc = could not find container \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": container with ID starting with 113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140 not found: ID does not exist" Jul 28 17:20:07.196429 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196283 2554 scope.go:117] "RemoveContainer" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.196528 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196504 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} err="failed to get container status \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": rpc error: code = NotFound desc = could not find container \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": container with ID starting with 795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109 not found: ID does not exist" Jul 28 17:20:07.196562 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196530 2554 scope.go:117] "RemoveContainer" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.196759 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196741 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} err="failed to get container status \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": rpc error: code = NotFound desc = could not find container \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": container with ID starting with ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8 not found: ID does not exist" Jul 28 17:20:07.196795 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196760 2554 scope.go:117] "RemoveContainer" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.196992 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196975 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} err="failed to get container status \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": rpc error: code = NotFound desc = could not find container \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": container with ID starting with 2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029 not found: ID does not exist" Jul 28 17:20:07.197050 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.196992 2554 scope.go:117] "RemoveContainer" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.197197 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197183 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624"} err="failed to get container status \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": rpc error: code = NotFound desc = could not find container \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": container with ID starting with b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624 not found: ID does not exist" Jul 28 17:20:07.197256 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197209 2554 scope.go:117] "RemoveContainer" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.197431 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197414 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} err="failed to get container status \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": rpc error: code = NotFound desc = could not find container \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": container with ID starting with d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6 not found: ID does not exist" Jul 28 17:20:07.197473 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197432 2554 scope.go:117] "RemoveContainer" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.197654 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197638 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} err="failed to get container status \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": rpc error: code = NotFound desc = could not find container \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": container with ID starting with fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496 not found: ID does not exist" Jul 28 17:20:07.197654 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197653 2554 scope.go:117] "RemoveContainer" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.197867 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197848 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} err="failed to get container status \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": rpc error: code = NotFound desc = could not find container \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": container with ID starting with 113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140 not found: ID does not exist" Jul 28 17:20:07.197937 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.197868 2554 scope.go:117] "RemoveContainer" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.198126 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198109 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} err="failed to get container status \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": rpc error: code = NotFound desc = could not find container \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": container with ID starting with 795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109 not found: ID does not exist" Jul 28 17:20:07.198178 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198126 2554 scope.go:117] "RemoveContainer" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.198324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198307 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} err="failed to get container status \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": rpc error: code = NotFound desc = could not find container \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": container with ID starting with ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8 not found: ID does not exist" Jul 28 17:20:07.198359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198325 2554 scope.go:117] "RemoveContainer" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.198536 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198518 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} err="failed to get container status \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": rpc error: code = NotFound desc = could not find container \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": container with ID starting with 2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029 not found: ID does not exist" Jul 28 17:20:07.198601 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198538 2554 scope.go:117] "RemoveContainer" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.198732 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198715 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624"} err="failed to get container status \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": rpc error: code = NotFound desc = could not find container \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": container with ID starting with b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624 not found: ID does not exist" Jul 28 17:20:07.198780 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198733 2554 scope.go:117] "RemoveContainer" containerID="d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6" Jul 28 17:20:07.198906 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198867 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6"} err="failed to get container status \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": rpc error: code = NotFound desc = could not find container \"d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6\": container with ID starting with d6fec35359c32014a390c816e6ae4fde1e5a92696c7796075f8de89c20adeba6 not found: ID does not exist" Jul 28 17:20:07.198954 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.198907 2554 scope.go:117] "RemoveContainer" containerID="fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496" Jul 28 17:20:07.199120 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199103 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496"} err="failed to get container status \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": rpc error: code = NotFound desc = could not find container \"fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496\": container with ID starting with fe6cc1f19296373531ac1b5410a28e6f7b9ee23d291d88029413f0ee11754496 not found: ID does not exist" Jul 28 17:20:07.199180 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199122 2554 scope.go:117] "RemoveContainer" containerID="113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140" Jul 28 17:20:07.199341 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199321 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140"} err="failed to get container status \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": rpc error: code = NotFound desc = could not find container \"113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140\": container with ID starting with 113cd0817b745b979e80e71eb4a39351be2e317e885cd83524e1f558bb322140 not found: ID does not exist" Jul 28 17:20:07.199341 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199339 2554 scope.go:117] "RemoveContainer" containerID="795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109" Jul 28 17:20:07.199572 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199553 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109"} err="failed to get container status \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": rpc error: code = NotFound desc = could not find container \"795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109\": container with ID starting with 795847c1480cc85645bc9488d79f4cd745a0cc1579522f4d3e90d2b59a4bb109 not found: ID does not exist" Jul 28 17:20:07.199637 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199573 2554 scope.go:117] "RemoveContainer" containerID="ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8" Jul 28 17:20:07.199637 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199594 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.199825 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199797 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8"} err="failed to get container status \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": rpc error: code = NotFound desc = could not find container \"ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8\": container with ID starting with ccfaf72210aaa4c3326882b0ed37a6e7a27436f0fb247f2d6768cf06e72ae4c8 not found: ID does not exist" Jul 28 17:20:07.199957 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.199826 2554 scope.go:117] "RemoveContainer" containerID="2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029" Jul 28 17:20:07.200272 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.200245 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029"} err="failed to get container status \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": rpc error: code = NotFound desc = could not find container \"2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029\": container with ID starting with 2128a8bb55c9a8451da9a43172a5b40c5bb91818c9ca7df915b0d52ac5fc1029 not found: ID does not exist" Jul 28 17:20:07.200272 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.200270 2554 scope.go:117] "RemoveContainer" containerID="b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624" Jul 28 17:20:07.200496 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.200478 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624"} err="failed to get container status \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": rpc error: code = NotFound desc = could not find container \"b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624\": container with ID starting with b19cc0c8b5bc95ad155a0e2f6a0055b03b88605d1e31a444a538d890d896b624 not found: ID does not exist" Jul 28 17:20:07.202810 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.202793 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 28 17:20:07.202810 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.202800 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 28 17:20:07.203019 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203005 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 28 17:20:07.203119 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203105 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 28 17:20:07.203177 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203163 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 28 17:20:07.203242 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203226 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 28 17:20:07.203354 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203339 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 28 17:20:07.203412 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203393 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2sdfb\"" Jul 28 17:20:07.203463 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203434 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 28 17:20:07.203524 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.203509 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 28 17:20:07.204092 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.204078 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 28 17:20:07.204173 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.204157 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 28 17:20:07.204345 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.204330 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d5i3tmhjt30lb\"" Jul 28 17:20:07.204709 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.204695 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 28 17:20:07.206766 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.206749 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 28 17:20:07.209141 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.209124 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 28 17:20:07.216257 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.216237 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:20:07.274988 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.274962 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.274988 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.274986 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275099 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275003 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275099 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275019 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-config\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275099 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275051 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-web-config\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275179 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275104 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19a8081d-9516-4560-b605-4891c9a02af5-config-out\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275179 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275120 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275179 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275139 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275179 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275170 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275199 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275225 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275244 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275264 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275277 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275325 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275343 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19a8081d-9516-4560-b605-4891c9a02af5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275362 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275380 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xw5s\" (UniqueName: \"kubernetes.io/projected/19a8081d-9516-4560-b605-4891c9a02af5-kube-api-access-2xw5s\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275423 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275404 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.275551 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.275422 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376326 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376268 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-web-config\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376326 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376303 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19a8081d-9516-4560-b605-4891c9a02af5-config-out\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376336 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376365 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376391 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376420 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376458 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376487 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376540 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376565 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376596 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376625 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19a8081d-9516-4560-b605-4891c9a02af5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376658 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376678 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xw5s\" (UniqueName: \"kubernetes.io/projected/19a8081d-9516-4560-b605-4891c9a02af5-kube-api-access-2xw5s\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.376724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376717 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.377380 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376764 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.377380 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376802 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.377380 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376826 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.377380 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376852 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.377380 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.376906 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-config\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.380264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.379217 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19a8081d-9516-4560-b605-4891c9a02af5-config-out\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.380264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.379350 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-config\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.380264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.379360 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.380264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.379453 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.380264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.379517 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-web-config\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.380264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.379738 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.380264 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.379771 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.380331 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.380420 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.380441 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.380584 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.380809 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.381008 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.381066 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.381804 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.381694 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.382091 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.382070 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19a8081d-9516-4560-b605-4891c9a02af5-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.382452 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.382435 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.382581 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.382556 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/19a8081d-9516-4560-b605-4891c9a02af5-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.383883 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.383858 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/19a8081d-9516-4560-b605-4891c9a02af5-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.388849 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.388830 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xw5s\" (UniqueName: \"kubernetes.io/projected/19a8081d-9516-4560-b605-4891c9a02af5-kube-api-access-2xw5s\") pod \"prometheus-k8s-0\" (UID: \"19a8081d-9516-4560-b605-4891c9a02af5\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.509511 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.509482 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:07.637492 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:07.637467 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 28 17:20:07.638811 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:20:07.638785 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a8081d_9516_4560_b605_4891c9a02af5.slice/crio-e85084c519664dce4b74da82363b1871d5b9c0e64f80fd26aa41177ed0d63005 WatchSource:0}: Error finding container e85084c519664dce4b74da82363b1871d5b9c0e64f80fd26aa41177ed0d63005: Status 404 returned error can't find the container with id e85084c519664dce4b74da82363b1871d5b9c0e64f80fd26aa41177ed0d63005 Jul 28 17:20:08.141680 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:08.141655 2554 generic.go:356] "Generic (PLEG): container finished" podID="19a8081d-9516-4560-b605-4891c9a02af5" containerID="7bc06a837d26f30aaa41e781c7014e6407a9b87234c7ee64e3f9b5a0ca7c288f" exitCode=0 Jul 28 17:20:08.141843 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:08.141725 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerDied","Data":"7bc06a837d26f30aaa41e781c7014e6407a9b87234c7ee64e3f9b5a0ca7c288f"} Jul 28 17:20:08.141843 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:08.141775 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerStarted","Data":"e85084c519664dce4b74da82363b1871d5b9c0e64f80fd26aa41177ed0d63005"} Jul 28 17:20:08.342009 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:08.340715 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06362a90-e2ed-42d6-9f52-064a2156ce1b" path="/var/lib/kubelet/pods/06362a90-e2ed-42d6-9f52-064a2156ce1b/volumes" Jul 28 17:20:09.151469 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.151443 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerStarted","Data":"25ee044aec20f5b0d549d675955db2276da7c3f0a1cb4ec327b3a5814c79b43a"} Jul 28 17:20:09.151801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.151503 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerStarted","Data":"e13619ef353ff038686609db9534086838f03795366423d7cc280748e611d742"} Jul 28 17:20:09.151801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.151532 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerStarted","Data":"62abce068763eda9a50ec71f36194ec9ac24928fba4a9840701cb47684cde272"} Jul 28 17:20:09.151801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.151554 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerStarted","Data":"6f510863f002c6d85dd6f851828e2a172b4331301028f97f7d2b86bd3e40f0e9"} Jul 28 17:20:09.151801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.151580 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerStarted","Data":"e7e6f91c43b454809ba3f246fa52d3c4b0a9cf39e94071e3f033bb041da139c3"} Jul 28 17:20:09.151801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.151607 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"19a8081d-9516-4560-b605-4891c9a02af5","Type":"ContainerStarted","Data":"e77d3f5a189bba5fe8b0fd696f454511647297c258ff39bb3c84c4b7430c4d34"} Jul 28 17:20:09.184231 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.184186 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.16749092 podStartE2EDuration="2.184171054s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.680134ms" isStatefulPod=false podCreationTimestamp="2026-07-28 17:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:20:09.181503602 +0000 UTC m=+167.399545059" watchObservedRunningTime="2026-07-28 17:20:09.184171054 +0000 UTC m=+167.402212516" Jul 28 17:20:09.437696 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.437638 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:09.501526 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:09.501501 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:12.509981 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:12.509931 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:20:14.527500 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:14.527420 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:14.628862 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:14.628830 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:14.688838 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:14.688806 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:21.058139 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:21.058093 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:21.155584 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:21.155527 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:24.388502 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:24.388462 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:24.487530 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:24.487491 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:27.928937 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:27.928898 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:28.032223 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:28.032189 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:35.378017 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:35.377981 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:35.485848 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:35.485815 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:35.563923 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:35.563856 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:20:50.836313 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:20:50.836280 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:02.071146 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:02.071113 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:02.147890 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:02.147839 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:02.216265 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:02.216226 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:03.277641 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:03.277605 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:07.510664 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:07.510611 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:21:07.526088 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:07.526059 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:21:08.401205 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:08.401174 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 28 17:21:11.893970 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:11.893921 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:11.953496 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:11.953451 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:12.317283 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:12.317251 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:12.410539 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:12.410490 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:12.511313 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:12.511280 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:34.812329 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:34.812285 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:34.910840 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:34.910801 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:41.035156 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:41.035121 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:41.359233 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:41.359135 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:41.593790 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:41.593751 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:41.890093 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:41.890053 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:42.131799 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:42.131766 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:42.471613 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:42.471577 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:42.669438 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:42.669396 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:42.769676 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:42.769638 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:42.872847 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:42.872814 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:43.074049 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:43.073963 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:43.260618 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:43.260578 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:43.458764 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:43.458682 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:21:43.558449 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:21:43.558411 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:22:22.181198 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:22.181167 2554 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 17:22:57.674179 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.674142 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:22:57.739959 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.739917 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-vc6md"] Jul 28 17:22:57.742042 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.742025 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:57.745001 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.744968 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 17:22:57.745170 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.745053 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 17:22:57.746716 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.746693 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 28 17:22:57.746797 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.746693 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-7j5qg\"" Jul 28 17:22:57.754588 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.754559 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-vc6md"] Jul 28 17:22:57.778891 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.778828 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-cert\") pod \"kserve-controller-manager-85d8b8ddfc-vc6md\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:57.779079 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.778957 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn9m7\" (UniqueName: \"kubernetes.io/projected/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-kube-api-access-gn9m7\") pod \"kserve-controller-manager-85d8b8ddfc-vc6md\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:57.880293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.880254 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-cert\") pod \"kserve-controller-manager-85d8b8ddfc-vc6md\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:57.880514 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.880354 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn9m7\" (UniqueName: \"kubernetes.io/projected/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-kube-api-access-gn9m7\") pod \"kserve-controller-manager-85d8b8ddfc-vc6md\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:57.882780 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.882755 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-cert\") pod \"kserve-controller-manager-85d8b8ddfc-vc6md\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:57.891564 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:57.891533 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn9m7\" (UniqueName: \"kubernetes.io/projected/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-kube-api-access-gn9m7\") pod \"kserve-controller-manager-85d8b8ddfc-vc6md\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:58.052755 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:58.052718 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:22:58.188916 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:58.188841 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-vc6md"] Jul 28 17:22:58.191790 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:22:58.191761 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b5ed1b6_c471_4fe8_81c3_1883e9e6c24f.slice/crio-41103d00184728f692c583b9d7ffb8f5a2d5e1e6268c1ec1cc949167dceb870c WatchSource:0}: Error finding container 41103d00184728f692c583b9d7ffb8f5a2d5e1e6268c1ec1cc949167dceb870c: Status 404 returned error can't find the container with id 41103d00184728f692c583b9d7ffb8f5a2d5e1e6268c1ec1cc949167dceb870c Jul 28 17:22:58.193173 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:58.193152 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:22:58.818651 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:58.818601 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" event={"ID":"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f","Type":"ContainerStarted","Data":"41103d00184728f692c583b9d7ffb8f5a2d5e1e6268c1ec1cc949167dceb870c"} Jul 28 17:22:58.851223 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:22:58.851174 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:23:00.831256 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:00.831145 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" event={"ID":"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f","Type":"ContainerStarted","Data":"51037e86ee5c346bf61488c73e5a7655c181e302d68f5f27b1f911a023e9e17b"} Jul 28 17:23:00.831722 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:00.831276 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:23:00.849688 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:00.849624 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" podStartSLOduration=1.3545062749999999 podStartE2EDuration="3.849605368s" totalImagesPullingTime="2.495099093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:00.848643924 +0000 UTC m=+339.066685389" watchObservedRunningTime="2026-07-28 17:23:00.849605368 +0000 UTC m=+339.067646830" Jul 28 17:23:31.841721 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:31.841684 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:23:33.130698 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:33.130663 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:23:33.232954 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:33.232916 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:23:39.835324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.835287 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-vc6md"] Jul 28 17:23:39.835823 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.835492 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" podUID="4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" containerName="manager" containerID="cri-o://51037e86ee5c346bf61488c73e5a7655c181e302d68f5f27b1f911a023e9e17b" gracePeriod=10 Jul 28 17:23:39.862455 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.862411 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-d97vr"] Jul 28 17:23:39.865810 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.865783 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:39.873891 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.873843 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-d97vr"] Jul 28 17:23:39.956715 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.956676 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvxjg\" (UniqueName: \"kubernetes.io/projected/cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0-kube-api-access-fvxjg\") pod \"kserve-controller-manager-85d8b8ddfc-d97vr\" (UID: \"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:39.956911 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.956750 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0-cert\") pod \"kserve-controller-manager-85d8b8ddfc-d97vr\" (UID: \"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:39.986329 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.986293 2554 generic.go:356] "Generic (PLEG): container finished" podID="4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" containerID="51037e86ee5c346bf61488c73e5a7655c181e302d68f5f27b1f911a023e9e17b" exitCode=0 Jul 28 17:23:39.986536 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:39.986368 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" event={"ID":"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f","Type":"ContainerDied","Data":"51037e86ee5c346bf61488c73e5a7655c181e302d68f5f27b1f911a023e9e17b"} Jul 28 17:23:40.057288 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.057241 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0-cert\") pod \"kserve-controller-manager-85d8b8ddfc-d97vr\" (UID: \"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:40.057490 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.057317 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvxjg\" (UniqueName: \"kubernetes.io/projected/cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0-kube-api-access-fvxjg\") pod \"kserve-controller-manager-85d8b8ddfc-d97vr\" (UID: \"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:40.059780 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.059749 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0-cert\") pod \"kserve-controller-manager-85d8b8ddfc-d97vr\" (UID: \"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:40.070638 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.070611 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvxjg\" (UniqueName: \"kubernetes.io/projected/cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0-kube-api-access-fvxjg\") pod \"kserve-controller-manager-85d8b8ddfc-d97vr\" (UID: \"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:40.085964 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.085892 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:23:40.157916 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.157848 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-cert\") pod \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " Jul 28 17:23:40.157916 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.157907 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gn9m7\" (UniqueName: \"kubernetes.io/projected/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-kube-api-access-gn9m7\") pod \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\" (UID: \"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f\") " Jul 28 17:23:40.160066 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.160036 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-cert" (OuterVolumeSpecName: "cert") pod "4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" (UID: "4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:40.160155 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.160067 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-kube-api-access-gn9m7" (OuterVolumeSpecName: "kube-api-access-gn9m7") pod "4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" (UID: "4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f"). InnerVolumeSpecName "kube-api-access-gn9m7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:23:40.217290 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.217243 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:40.258946 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.258911 2554 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:23:40.258946 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.258941 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gn9m7\" (UniqueName: \"kubernetes.io/projected/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f-kube-api-access-gn9m7\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:23:40.351403 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.351367 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-d97vr"] Jul 28 17:23:40.353538 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:23:40.353511 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf7dc3f1_f8f5_4d3e_ade1_3d3767f23fe0.slice/crio-42d5e39a412d7ba86fe30b37707c2e76005911900f8d8b2bba9afa26ef4eec55 WatchSource:0}: Error finding container 42d5e39a412d7ba86fe30b37707c2e76005911900f8d8b2bba9afa26ef4eec55: Status 404 returned error can't find the container with id 42d5e39a412d7ba86fe30b37707c2e76005911900f8d8b2bba9afa26ef4eec55 Jul 28 17:23:40.992008 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.991915 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" Jul 28 17:23:40.992008 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.991937 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-vc6md" event={"ID":"4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f","Type":"ContainerDied","Data":"41103d00184728f692c583b9d7ffb8f5a2d5e1e6268c1ec1cc949167dceb870c"} Jul 28 17:23:40.992554 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.992016 2554 scope.go:117] "RemoveContainer" containerID="51037e86ee5c346bf61488c73e5a7655c181e302d68f5f27b1f911a023e9e17b" Jul 28 17:23:40.993683 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.993652 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" event={"ID":"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0","Type":"ContainerStarted","Data":"31ff2795ce7be720a157df3fd3cd5c07b9691777e096b7ec3b57fd26ad7e0bca"} Jul 28 17:23:40.993816 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.993707 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" event={"ID":"cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0","Type":"ContainerStarted","Data":"42d5e39a412d7ba86fe30b37707c2e76005911900f8d8b2bba9afa26ef4eec55"} Jul 28 17:23:40.993816 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:40.993730 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:23:41.014131 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:41.014079 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" podStartSLOduration=1.674541772 podStartE2EDuration="2.014061589s" totalImagesPullingTime="339.519817ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:41.013983494 +0000 UTC m=+379.232024957" watchObservedRunningTime="2026-07-28 17:23:41.014061589 +0000 UTC m=+379.232103053" Jul 28 17:23:41.033537 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:41.033498 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-vc6md"] Jul 28 17:23:41.036042 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:41.036007 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-vc6md"] Jul 28 17:23:42.336297 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:42.336262 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" path="/var/lib/kubelet/pods/4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f/volumes" Jul 28 17:23:59.631891 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.631843 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-856748c4fb-skff4"] Jul 28 17:23:59.632296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.632214 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" containerName="manager" Jul 28 17:23:59.632296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.632227 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" containerName="manager" Jul 28 17:23:59.632296 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.632286 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5ed1b6-c471-4fe8-81c3-1883e9e6c24f" containerName="manager" Jul 28 17:23:59.635238 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.635219 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.638293 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.638265 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 17:23:59.642801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.642778 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-66zv9\"" Jul 28 17:23:59.642944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.642809 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 17:23:59.642944 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.642904 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 17:23:59.643078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.642957 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 17:23:59.643078 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.642911 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 17:23:59.647247 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.647220 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 17:23:59.647633 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.647612 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-856748c4fb-skff4"] Jul 28 17:23:59.688366 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.688332 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:23:59.722841 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.722804 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5506192f-7633-4628-9559-b115d83bb42a-console-serving-cert\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.722841 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.722842 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-console-config\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.723064 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.722868 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5506192f-7633-4628-9559-b115d83bb42a-console-oauth-config\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.723064 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.722927 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldxj2\" (UniqueName: \"kubernetes.io/projected/5506192f-7633-4628-9559-b115d83bb42a-kube-api-access-ldxj2\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.723064 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.722959 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-oauth-serving-cert\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.723064 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.722977 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-service-ca\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.723204 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.723080 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-trusted-ca-bundle\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.823716 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.823678 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5506192f-7633-4628-9559-b115d83bb42a-console-serving-cert\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.823853 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.823717 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-console-config\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.823853 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.823740 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5506192f-7633-4628-9559-b115d83bb42a-console-oauth-config\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.823997 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.823858 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldxj2\" (UniqueName: \"kubernetes.io/projected/5506192f-7633-4628-9559-b115d83bb42a-kube-api-access-ldxj2\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.823997 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.823946 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-oauth-serving-cert\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.823997 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.823972 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-service-ca\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.824138 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.824014 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-trusted-ca-bundle\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.824590 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.824564 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-console-config\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.824724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.824646 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-service-ca\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.824724 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.824679 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-oauth-serving-cert\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.825320 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.825297 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5506192f-7633-4628-9559-b115d83bb42a-trusted-ca-bundle\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.826453 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.826435 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5506192f-7633-4628-9559-b115d83bb42a-console-oauth-config\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.826543 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.826524 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5506192f-7633-4628-9559-b115d83bb42a-console-serving-cert\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.834810 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.834787 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldxj2\" (UniqueName: \"kubernetes.io/projected/5506192f-7633-4628-9559-b115d83bb42a-kube-api-access-ldxj2\") pod \"console-856748c4fb-skff4\" (UID: \"5506192f-7633-4628-9559-b115d83bb42a\") " pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:23:59.945025 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:23:59.944919 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:24:00.084015 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:00.083987 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-856748c4fb-skff4"] Jul 28 17:24:00.086868 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:24:00.086837 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5506192f_7633_4628_9559_b115d83bb42a.slice/crio-8c59ad47b5c5b5f4be21fc91678823b6bf7d108dbf6e1c8126467bcef25c8897 WatchSource:0}: Error finding container 8c59ad47b5c5b5f4be21fc91678823b6bf7d108dbf6e1c8126467bcef25c8897: Status 404 returned error can't find the container with id 8c59ad47b5c5b5f4be21fc91678823b6bf7d108dbf6e1c8126467bcef25c8897 Jul 28 17:24:01.074145 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:01.074103 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-856748c4fb-skff4" event={"ID":"5506192f-7633-4628-9559-b115d83bb42a","Type":"ContainerStarted","Data":"ba0dc9a28d514e9bf08a21789f1f7977b44f9429640566dcd64e31aafbd428b1"} Jul 28 17:24:01.074573 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:01.074161 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-856748c4fb-skff4" event={"ID":"5506192f-7633-4628-9559-b115d83bb42a","Type":"ContainerStarted","Data":"8c59ad47b5c5b5f4be21fc91678823b6bf7d108dbf6e1c8126467bcef25c8897"} Jul 28 17:24:01.095146 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:01.095083 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-856748c4fb-skff4" podStartSLOduration=2.095058357 podStartE2EDuration="2.095058357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:01.093224762 +0000 UTC m=+399.311266226" watchObservedRunningTime="2026-07-28 17:24:01.095058357 +0000 UTC m=+399.313099885" Jul 28 17:24:09.945489 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:09.945450 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:24:09.945969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:09.945511 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:24:09.951294 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:09.951263 2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:24:10.116977 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:10.116946 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-856748c4fb-skff4" Jul 28 17:24:10.275862 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:10.275824 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:10.376200 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:10.376160 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:10.478611 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:10.478575 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:12.003309 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:12.003274 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-d97vr" Jul 28 17:24:13.442590 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:13.442550 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:13.572680 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:13.572642 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:13.674849 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:13.674812 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:13.817811 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:13.817766 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:14.148832 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:14.148738 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:14.249771 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:14.249737 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:24:28.848300 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.848259 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-tvxnm"] Jul 28 17:24:28.852807 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.852786 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tvxnm" Jul 28 17:24:28.856191 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.856160 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-j2xjc\"" Jul 28 17:24:28.856191 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.856160 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 28 17:24:28.862375 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.862346 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tvxnm"] Jul 28 17:24:28.884985 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.884952 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbmrh\" (UniqueName: \"kubernetes.io/projected/3b770d99-7056-4944-81bf-5c31df331e4a-kube-api-access-bbmrh\") pod \"s3-init-tvxnm\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " pod="kserve/s3-init-tvxnm" Jul 28 17:24:28.885155 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.885046 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b770d99-7056-4944-81bf-5c31df331e4a-model-cache\") pod \"s3-init-tvxnm\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " pod="kserve/s3-init-tvxnm" Jul 28 17:24:28.985995 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.985955 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbmrh\" (UniqueName: \"kubernetes.io/projected/3b770d99-7056-4944-81bf-5c31df331e4a-kube-api-access-bbmrh\") pod \"s3-init-tvxnm\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " pod="kserve/s3-init-tvxnm" Jul 28 17:24:28.986180 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.986030 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b770d99-7056-4944-81bf-5c31df331e4a-model-cache\") pod \"s3-init-tvxnm\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " pod="kserve/s3-init-tvxnm" Jul 28 17:24:28.986438 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.986416 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b770d99-7056-4944-81bf-5c31df331e4a-model-cache\") pod \"s3-init-tvxnm\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " pod="kserve/s3-init-tvxnm" Jul 28 17:24:28.995810 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:28.995771 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbmrh\" (UniqueName: \"kubernetes.io/projected/3b770d99-7056-4944-81bf-5c31df331e4a-kube-api-access-bbmrh\") pod \"s3-init-tvxnm\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " pod="kserve/s3-init-tvxnm" Jul 28 17:24:29.180800 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:29.180704 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tvxnm" Jul 28 17:24:29.318972 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:29.318945 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-tvxnm"] Jul 28 17:24:29.321535 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:24:29.321498 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b770d99_7056_4944_81bf_5c31df331e4a.slice/crio-673db4aa48fc0c6a32ee5a65c966a38be5e7ab95112bb7dc6aefc5d4232c8f5e WatchSource:0}: Error finding container 673db4aa48fc0c6a32ee5a65c966a38be5e7ab95112bb7dc6aefc5d4232c8f5e: Status 404 returned error can't find the container with id 673db4aa48fc0c6a32ee5a65c966a38be5e7ab95112bb7dc6aefc5d4232c8f5e Jul 28 17:24:30.201479 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:30.201438 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tvxnm" event={"ID":"3b770d99-7056-4944-81bf-5c31df331e4a","Type":"ContainerStarted","Data":"673db4aa48fc0c6a32ee5a65c966a38be5e7ab95112bb7dc6aefc5d4232c8f5e"} Jul 28 17:24:33.216671 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:33.216625 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tvxnm" event={"ID":"3b770d99-7056-4944-81bf-5c31df331e4a","Type":"ContainerStarted","Data":"43e5487ea840eddfb09ad43b4b8aefff56b6742af9e4051ba4e88837bde60b5f"} Jul 28 17:24:46.271483 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:46.271442 2554 generic.go:356] "Generic (PLEG): container finished" podID="3b770d99-7056-4944-81bf-5c31df331e4a" containerID="43e5487ea840eddfb09ad43b4b8aefff56b6742af9e4051ba4e88837bde60b5f" exitCode=0 Jul 28 17:24:46.272062 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:46.271516 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tvxnm" event={"ID":"3b770d99-7056-4944-81bf-5c31df331e4a","Type":"ContainerDied","Data":"43e5487ea840eddfb09ad43b4b8aefff56b6742af9e4051ba4e88837bde60b5f"} Jul 28 17:24:51.298335 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:51.298283 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tvxnm" event={"ID":"3b770d99-7056-4944-81bf-5c31df331e4a","Type":"ContainerStarted","Data":"6be37f5b4381c204a6a029160ce37f606735b95edc2134d86a06d0c918c7adfd"} Jul 28 17:24:51.317681 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:24:51.317625 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-tvxnm" podStartSLOduration=2.839092371 podStartE2EDuration="23.31760758s" totalImagesPullingTime="7.542352435s" totalInitContainerRuntime="12.936162774s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:51.316731546 +0000 UTC m=+449.534773013" watchObservedRunningTime="2026-07-28 17:24:51.31760758 +0000 UTC m=+449.535649042" Jul 28 17:25:41.994981 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:41.994941 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:25:56.571978 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:56.571938 2554 generic.go:356] "Generic (PLEG): container finished" podID="3b770d99-7056-4944-81bf-5c31df331e4a" containerID="6be37f5b4381c204a6a029160ce37f606735b95edc2134d86a06d0c918c7adfd" exitCode=0 Jul 28 17:25:56.572332 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:56.572012 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tvxnm" event={"ID":"3b770d99-7056-4944-81bf-5c31df331e4a","Type":"ContainerDied","Data":"6be37f5b4381c204a6a029160ce37f606735b95edc2134d86a06d0c918c7adfd"} Jul 28 17:25:56.851697 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:56.851634 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:25:57.698278 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:57.698255 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tvxnm" Jul 28 17:25:57.772389 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:57.772367 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbmrh\" (UniqueName: \"kubernetes.io/projected/3b770d99-7056-4944-81bf-5c31df331e4a-kube-api-access-bbmrh\") pod \"3b770d99-7056-4944-81bf-5c31df331e4a\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " Jul 28 17:25:57.772489 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:57.772401 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b770d99-7056-4944-81bf-5c31df331e4a-model-cache\") pod \"3b770d99-7056-4944-81bf-5c31df331e4a\" (UID: \"3b770d99-7056-4944-81bf-5c31df331e4a\") " Jul 28 17:25:57.774333 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:57.774307 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b770d99-7056-4944-81bf-5c31df331e4a-kube-api-access-bbmrh" (OuterVolumeSpecName: "kube-api-access-bbmrh") pod "3b770d99-7056-4944-81bf-5c31df331e4a" (UID: "3b770d99-7056-4944-81bf-5c31df331e4a"). InnerVolumeSpecName "kube-api-access-bbmrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:25:57.791051 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:57.791026 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b770d99-7056-4944-81bf-5c31df331e4a-model-cache" (OuterVolumeSpecName: "model-cache") pod "3b770d99-7056-4944-81bf-5c31df331e4a" (UID: "3b770d99-7056-4944-81bf-5c31df331e4a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:25:57.873805 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:57.873752 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbmrh\" (UniqueName: \"kubernetes.io/projected/3b770d99-7056-4944-81bf-5c31df331e4a-kube-api-access-bbmrh\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:25:57.873805 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:57.873774 2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3b770d99-7056-4944-81bf-5c31df331e4a-model-cache\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:25:58.580645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:58.580622 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-tvxnm" Jul 28 17:25:58.580645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:58.580634 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-tvxnm" event={"ID":"3b770d99-7056-4944-81bf-5c31df331e4a","Type":"ContainerDied","Data":"673db4aa48fc0c6a32ee5a65c966a38be5e7ab95112bb7dc6aefc5d4232c8f5e"} Jul 28 17:25:58.580794 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:58.580661 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="673db4aa48fc0c6a32ee5a65c966a38be5e7ab95112bb7dc6aefc5d4232c8f5e" Jul 28 17:25:58.969026 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:58.968994 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:25:59.069977 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:25:59.069950 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:09.629565 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:09.629524 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:09.695292 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:09.695242 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:11.562629 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.562597 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:11.666060 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.666029 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr"] Jul 28 17:26:11.666348 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.666334 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b770d99-7056-4944-81bf-5c31df331e4a" containerName="s3-init" Jul 28 17:26:11.666395 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.666350 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b770d99-7056-4944-81bf-5c31df331e4a" containerName="s3-init" Jul 28 17:26:11.666395 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.666361 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b770d99-7056-4944-81bf-5c31df331e4a" containerName="download-hf-model" Jul 28 17:26:11.666395 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.666367 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b770d99-7056-4944-81bf-5c31df331e4a" containerName="download-hf-model" Jul 28 17:26:11.666555 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.666428 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b770d99-7056-4944-81bf-5c31df331e4a" containerName="s3-init" Jul 28 17:26:11.668783 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.668768 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.671777 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.671752 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-tdp4m\"" Jul 28 17:26:11.671777 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.671768 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 28 17:26:11.671964 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.671768 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 28 17:26:11.673124 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.673110 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 28 17:26:11.673198 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.673156 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:26:11.673198 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.673181 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 28 17:26:11.673332 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.673224 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 28 17:26:11.681792 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.681770 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr"] Jul 28 17:26:11.763151 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.763120 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:11.772360 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.772332 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2t8p\" (UniqueName: \"kubernetes.io/projected/149c3c47-5c42-4518-a006-4977e053d6f1-kube-api-access-r2t8p\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.772497 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.772381 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.772497 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.772421 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/149c3c47-5c42-4518-a006-4977e053d6f1-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.772497 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.772484 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/149c3c47-5c42-4518-a006-4977e053d6f1-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.772673 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.772546 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.862185 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.862116 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:11.873100 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.873080 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.873196 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.873110 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/149c3c47-5c42-4518-a006-4977e053d6f1-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.873196 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.873146 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/149c3c47-5c42-4518-a006-4977e053d6f1-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.873196 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.873175 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.873331 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.873202 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2t8p\" (UniqueName: \"kubernetes.io/projected/149c3c47-5c42-4518-a006-4977e053d6f1-kube-api-access-r2t8p\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.873634 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.873617 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/149c3c47-5c42-4518-a006-4977e053d6f1-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.873971 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.873950 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.874072 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.874057 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.875689 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.875664 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/149c3c47-5c42-4518-a006-4977e053d6f1-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.883306 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.883285 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2t8p\" (UniqueName: \"kubernetes.io/projected/149c3c47-5c42-4518-a006-4977e053d6f1-kube-api-access-r2t8p\") pod \"isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:11.887669 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.887646 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb"] Jul 28 17:26:11.890201 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.890185 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:11.893066 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.893045 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 28 17:26:11.893165 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.893131 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 28 17:26:11.902217 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.902041 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb"] Jul 28 17:26:11.960498 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.960450 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:11.973538 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.973516 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8245\" (UniqueName: \"kubernetes.io/projected/c37a984c-db24-4b00-aa53-82cfe0722356-kube-api-access-q8245\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:11.973645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.973559 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c37a984c-db24-4b00-aa53-82cfe0722356-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:11.973645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.973585 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:11.973716 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.973637 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c37a984c-db24-4b00-aa53-82cfe0722356-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:11.973716 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.973697 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:11.979510 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:11.979490 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:12.033981 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.033944 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:12.074474 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.074417 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c37a984c-db24-4b00-aa53-82cfe0722356-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.074598 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.074514 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.074598 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.074566 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8245\" (UniqueName: \"kubernetes.io/projected/c37a984c-db24-4b00-aa53-82cfe0722356-kube-api-access-q8245\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.074669 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.074622 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c37a984c-db24-4b00-aa53-82cfe0722356-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.074669 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.074659 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.075305 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.075255 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c37a984c-db24-4b00-aa53-82cfe0722356-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.075430 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.075354 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.075430 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.075401 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.076982 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.076961 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c37a984c-db24-4b00-aa53-82cfe0722356-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.086111 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.086084 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd"] Jul 28 17:26:12.087742 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.087721 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8245\" (UniqueName: \"kubernetes.io/projected/c37a984c-db24-4b00-aa53-82cfe0722356-kube-api-access-q8245\") pod \"isvc-xgboost-graph-predictor-555fc69d56-8fhcb\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.089976 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.089958 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.093053 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.093035 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 28 17:26:12.093154 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.093040 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 28 17:26:12.101066 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.101042 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd"] Jul 28 17:26:12.115018 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.114960 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:12.115483 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.115463 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr"] Jul 28 17:26:12.117786 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:26:12.117765 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod149c3c47_5c42_4518_a006_4977e053d6f1.slice/crio-49a43239ceb84baa9b5c6315ae0efea463801d92bdde3f35533942d8421f50d3 WatchSource:0}: Error finding container 49a43239ceb84baa9b5c6315ae0efea463801d92bdde3f35533942d8421f50d3: Status 404 returned error can't find the container with id 49a43239ceb84baa9b5c6315ae0efea463801d92bdde3f35533942d8421f50d3 Jul 28 17:26:12.175068 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.175048 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgm2c\" (UniqueName: \"kubernetes.io/projected/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-api-access-bgm2c\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.175138 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.175090 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.175138 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.175126 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.175242 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.175179 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.175294 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.175247 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.201571 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.201532 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:12.276511 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.276484 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgm2c\" (UniqueName: \"kubernetes.io/projected/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-api-access-bgm2c\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.276650 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.276525 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.276650 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.276555 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.276650 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.276597 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.276812 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.276652 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.276812 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:26:12.276736 2554 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-serving-cert: secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 28 17:26:12.276812 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:26:12.276791 2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls podName:4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff nodeName:}" failed. No retries permitted until 2026-07-28 17:26:12.776775201 +0000 UTC m=+530.994816641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls") pod "isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" (UID: "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff") : secret "isvc-sklearn-graph-2-predictor-serving-cert" not found Jul 28 17:26:12.277014 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.276993 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.277495 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.277472 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.277700 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.277668 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.286215 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.285911 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgm2c\" (UniqueName: \"kubernetes.io/projected/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-api-access-bgm2c\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.306358 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.306324 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:12.341272 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.341227 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb"] Jul 28 17:26:12.343432 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:26:12.343408 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc37a984c_db24_4b00_aa53_82cfe0722356.slice/crio-589823f1cfe47474547461a078a9c0e6891a6ef4f87244f29fbe79f52d4a2641 WatchSource:0}: Error finding container 589823f1cfe47474547461a078a9c0e6891a6ef4f87244f29fbe79f52d4a2641: Status 404 returned error can't find the container with id 589823f1cfe47474547461a078a9c0e6891a6ef4f87244f29fbe79f52d4a2641 Jul 28 17:26:12.404176 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.404155 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:12.637510 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.637427 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerStarted","Data":"32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6"} Jul 28 17:26:12.637510 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.637503 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerStarted","Data":"589823f1cfe47474547461a078a9c0e6891a6ef4f87244f29fbe79f52d4a2641"} Jul 28 17:26:12.639206 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.639174 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerStarted","Data":"a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00"} Jul 28 17:26:12.639329 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.639240 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerStarted","Data":"49a43239ceb84baa9b5c6315ae0efea463801d92bdde3f35533942d8421f50d3"} Jul 28 17:26:12.782394 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.782358 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:12.785059 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:12.785032 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7d44765f95-b55fd\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:13.000128 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:13.000105 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:13.129559 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:13.129536 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd"] Jul 28 17:26:13.131728 ip-10-0-129-206 kubenswrapper[2554]: W0728 17:26:13.131706 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae5b6aa_0fb0_4cfa_ba5c_838123a8d2ff.slice/crio-81587563b8f0d734f8310c7c16da083da99ac8684d9d76715ea7bcb9bc1d6aa0 WatchSource:0}: Error finding container 81587563b8f0d734f8310c7c16da083da99ac8684d9d76715ea7bcb9bc1d6aa0: Status 404 returned error can't find the container with id 81587563b8f0d734f8310c7c16da083da99ac8684d9d76715ea7bcb9bc1d6aa0 Jul 28 17:26:13.646324 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:13.646294 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerStarted","Data":"f7352fe91927e6ac08bbbe34818780a78c79696cac72cab3510dd397c2df4bc3"} Jul 28 17:26:13.646755 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:13.646353 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerStarted","Data":"81587563b8f0d734f8310c7c16da083da99ac8684d9d76715ea7bcb9bc1d6aa0"} Jul 28 17:26:16.661946 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:16.661841 2554 generic.go:356] "Generic (PLEG): container finished" podID="149c3c47-5c42-4518-a006-4977e053d6f1" containerID="a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00" exitCode=0 Jul 28 17:26:16.662359 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:16.661934 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerDied","Data":"a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00"} Jul 28 17:26:16.663505 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:16.663470 2554 generic.go:356] "Generic (PLEG): container finished" podID="c37a984c-db24-4b00-aa53-82cfe0722356" containerID="32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6" exitCode=0 Jul 28 17:26:16.663585 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:16.663527 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerDied","Data":"32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6"} Jul 28 17:26:17.677001 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:17.676967 2554 generic.go:356] "Generic (PLEG): container finished" podID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerID="f7352fe91927e6ac08bbbe34818780a78c79696cac72cab3510dd397c2df4bc3" exitCode=0 Jul 28 17:26:17.677477 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:17.677010 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerDied","Data":"f7352fe91927e6ac08bbbe34818780a78c79696cac72cab3510dd397c2df4bc3"} Jul 28 17:26:27.964397 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:27.964352 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:32.804403 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:32.804362 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:44.852708 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:44.852602 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerStarted","Data":"6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79"} Jul 28 17:26:44.855516 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:44.855437 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerStarted","Data":"0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e"} Jul 28 17:26:44.860726 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:44.860543 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerStarted","Data":"a827fe3367efe90323cca9f70055a476e184357b854f17022910c85bf58417cc"} Jul 28 17:26:47.889378 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.889337 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerStarted","Data":"f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8"} Jul 28 17:26:47.889925 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.889562 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:47.889925 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.889613 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:47.891409 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.891377 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:26:47.891925 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.891900 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerStarted","Data":"022d73cebb7a2c5a61c7f139b42ceaac5e653dadedeae6ed94fee079060eadea"} Jul 28 17:26:47.892081 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.892057 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:47.892170 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.892118 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:47.893381 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.893347 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:26:47.894326 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.894302 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerStarted","Data":"1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f"} Jul 28 17:26:47.894545 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.894516 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:47.894623 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.894569 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:47.895539 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.895514 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:26:47.910892 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.910809 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podStartSLOduration=2.5871166949999997 podStartE2EDuration="36.910789497s" totalImagesPullingTime="30.412923488s" totalInitContainerRuntime="3.910749314s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:26:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:26:47.909957386 +0000 UTC m=+566.127998850" watchObservedRunningTime="2026-07-28 17:26:47.910789497 +0000 UTC m=+566.128830960" Jul 28 17:26:47.932704 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.932645 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podStartSLOduration=2.702517159 podStartE2EDuration="35.932627318s" totalImagesPullingTime="29.397249436s" totalInitContainerRuntime="3.832860723s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:26:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:26:47.931987519 +0000 UTC m=+566.150028985" watchObservedRunningTime="2026-07-28 17:26:47.932627318 +0000 UTC m=+566.150668781" Jul 28 17:26:47.953575 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:47.953504 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podStartSLOduration=2.9653801509999997 podStartE2EDuration="36.953484515s" totalImagesPullingTime="30.191550801s" totalInitContainerRuntime="3.796553563s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:26:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:26:47.95167448 +0000 UTC m=+566.169715944" watchObservedRunningTime="2026-07-28 17:26:47.953484515 +0000 UTC m=+566.171525977" Jul 28 17:26:48.899781 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:48.899736 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:26:48.900265 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:48.899833 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:26:48.900265 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:48.899833 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:26:52.493736 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:52.493694 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:52.569145 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:52.569113 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:26:53.904635 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:53.904596 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:26:53.905363 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:53.905313 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:26:53.905739 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:53.905721 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:26:53.905851 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:53.905814 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:26:53.906281 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:53.906243 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:26:53.906281 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:26:53.906258 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:27:03.905592 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:03.905542 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:27:03.906236 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:03.906213 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:27:03.906304 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:03.906213 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:27:13.905590 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:13.905550 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:27:13.906790 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:13.906763 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:27:13.906790 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:13.906777 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:27:23.905993 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:23.905944 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:27:23.906468 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:23.906276 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:27:23.906468 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:23.906276 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:27:31.582596 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:31.582556 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:32.026035 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:32.025990 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:33.906184 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:33.906141 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:27:33.906655 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:33.906319 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:27:33.906655 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:33.906347 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:27:43.905743 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:43.905700 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:27:43.906276 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:43.906159 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:27:43.906276 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:43.906224 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:27:45.761937 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:45.761856 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:45.882028 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:45.881988 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:45.982564 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:45.982526 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:46.059000 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:46.058904 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:46.374999 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:46.374899 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:46.475474 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:46.475438 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:27:53.906036 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:53.906005 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:27:53.906516 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:53.906186 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:27:53.906516 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:53.906258 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:27:59.332340 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:27:59.332288 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.29:8080: connect: connection refused" Jul 28 17:28:03.907109 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:03.907067 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.27:8080: connect: connection refused" Jul 28 17:28:09.333134 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:09.333091 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:28:09.333962 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:09.333938 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:28:31.562534 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:31.562488 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:32.808950 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:32.808904 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:41.686558 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.686519 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:41.786303 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.786265 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb"] Jul 28 17:28:41.786547 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.786525 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" containerID="cri-o://0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e" gracePeriod=30 Jul 28 17:28:41.786619 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.786568 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kube-rbac-proxy" containerID="cri-o://f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8" gracePeriod=30 Jul 28 17:28:41.815051 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.815018 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:41.848800 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.848759 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd"] Jul 28 17:28:41.849090 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.849061 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" containerID="cri-o://a827fe3367efe90323cca9f70055a476e184357b854f17022910c85bf58417cc" gracePeriod=30 Jul 28 17:28:41.849198 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.849079 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kube-rbac-proxy" containerID="cri-o://022d73cebb7a2c5a61c7f139b42ceaac5e653dadedeae6ed94fee079060eadea" gracePeriod=30 Jul 28 17:28:41.901137 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.901103 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:41.911599 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.911561 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr"] Jul 28 17:28:41.911824 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.911794 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" containerID="cri-o://6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79" gracePeriod=30 Jul 28 17:28:41.911969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.911827 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kube-rbac-proxy" containerID="cri-o://1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f" gracePeriod=30 Jul 28 17:28:41.981577 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:41.981536 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:42.137893 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.137849 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:42.302354 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.302315 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:42.455567 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.455529 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:42.486141 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.486083 2554 generic.go:356] "Generic (PLEG): container finished" podID="149c3c47-5c42-4518-a006-4977e053d6f1" containerID="1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f" exitCode=2 Jul 28 17:28:42.486141 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.486127 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerDied","Data":"1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f"} Jul 28 17:28:42.488596 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.488561 2554 generic.go:356] "Generic (PLEG): container finished" podID="c37a984c-db24-4b00-aa53-82cfe0722356" containerID="f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8" exitCode=2 Jul 28 17:28:42.488774 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.488639 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerDied","Data":"f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8"} Jul 28 17:28:42.490818 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.490789 2554 generic.go:356] "Generic (PLEG): container finished" podID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerID="022d73cebb7a2c5a61c7f139b42ceaac5e653dadedeae6ed94fee079060eadea" exitCode=2 Jul 28 17:28:42.490984 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:42.490863 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerDied","Data":"022d73cebb7a2c5a61c7f139b42ceaac5e653dadedeae6ed94fee079060eadea"} Jul 28 17:28:43.900066 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:43.900024 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.28:8643/healthz\": dial tcp 10.134.0.28:8643: connect: connection refused" Jul 28 17:28:43.900488 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:43.900024 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.27:8643/healthz\": dial tcp 10.134.0.27:8643: connect: connection refused" Jul 28 17:28:43.900488 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:43.900024 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.29:8643/healthz\": dial tcp 10.134.0.29:8643: connect: connection refused" Jul 28 17:28:43.905690 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:43.905652 2554 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.28:8080: connect: connection refused" Jul 28 17:28:46.007166 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.007132 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:46.252073 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.252038 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:28:46.324670 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.324561 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-kube-rbac-proxy-sar-config\") pod \"c37a984c-db24-4b00-aa53-82cfe0722356\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " Jul 28 17:28:46.324670 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.324610 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c37a984c-db24-4b00-aa53-82cfe0722356-kserve-provision-location\") pod \"c37a984c-db24-4b00-aa53-82cfe0722356\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " Jul 28 17:28:46.324670 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.324672 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8245\" (UniqueName: \"kubernetes.io/projected/c37a984c-db24-4b00-aa53-82cfe0722356-kube-api-access-q8245\") pod \"c37a984c-db24-4b00-aa53-82cfe0722356\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " Jul 28 17:28:46.324996 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.324705 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c37a984c-db24-4b00-aa53-82cfe0722356-proxy-tls\") pod \"c37a984c-db24-4b00-aa53-82cfe0722356\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " Jul 28 17:28:46.324996 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.324778 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-cabundle-cert\") pod \"c37a984c-db24-4b00-aa53-82cfe0722356\" (UID: \"c37a984c-db24-4b00-aa53-82cfe0722356\") " Jul 28 17:28:46.325095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.324986 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c37a984c-db24-4b00-aa53-82cfe0722356-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c37a984c-db24-4b00-aa53-82cfe0722356" (UID: "c37a984c-db24-4b00-aa53-82cfe0722356"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:28:46.325163 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.325117 2554 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c37a984c-db24-4b00-aa53-82cfe0722356-kserve-provision-location\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:46.325163 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.325051 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c37a984c-db24-4b00-aa53-82cfe0722356" (UID: "c37a984c-db24-4b00-aa53-82cfe0722356"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:28:46.325273 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.325188 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c37a984c-db24-4b00-aa53-82cfe0722356" (UID: "c37a984c-db24-4b00-aa53-82cfe0722356"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:28:46.327263 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.327230 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c37a984c-db24-4b00-aa53-82cfe0722356-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c37a984c-db24-4b00-aa53-82cfe0722356" (UID: "c37a984c-db24-4b00-aa53-82cfe0722356"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:28:46.327400 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.327275 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c37a984c-db24-4b00-aa53-82cfe0722356-kube-api-access-q8245" (OuterVolumeSpecName: "kube-api-access-q8245") pod "c37a984c-db24-4b00-aa53-82cfe0722356" (UID: "c37a984c-db24-4b00-aa53-82cfe0722356"). InnerVolumeSpecName "kube-api-access-q8245". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:28:46.425952 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.425909 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c37a984c-db24-4b00-aa53-82cfe0722356-proxy-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:46.425952 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.425943 2554 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-cabundle-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:46.425952 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.425953 2554 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c37a984c-db24-4b00-aa53-82cfe0722356-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:46.425952 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.425962 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8245\" (UniqueName: \"kubernetes.io/projected/c37a984c-db24-4b00-aa53-82cfe0722356-kube-api-access-q8245\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:46.515891 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.515829 2554 generic.go:356] "Generic (PLEG): container finished" podID="c37a984c-db24-4b00-aa53-82cfe0722356" containerID="0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e" exitCode=0 Jul 28 17:28:46.516095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.515918 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" Jul 28 17:28:46.516095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.515917 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerDied","Data":"0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e"} Jul 28 17:28:46.516095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.515989 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb" event={"ID":"c37a984c-db24-4b00-aa53-82cfe0722356","Type":"ContainerDied","Data":"589823f1cfe47474547461a078a9c0e6891a6ef4f87244f29fbe79f52d4a2641"} Jul 28 17:28:46.516095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.516022 2554 scope.go:117] "RemoveContainer" containerID="f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8" Jul 28 17:28:46.525421 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.525393 2554 scope.go:117] "RemoveContainer" containerID="0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e" Jul 28 17:28:46.526732 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.526707 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:28:46.534125 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.534103 2554 scope.go:117] "RemoveContainer" containerID="32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6" Jul 28 17:28:46.542445 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.542404 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb"] Jul 28 17:28:46.543242 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.543224 2554 scope.go:117] "RemoveContainer" containerID="f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8" Jul 28 17:28:46.543688 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:28:46.543667 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8\": container with ID starting with f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8 not found: ID does not exist" containerID="f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8" Jul 28 17:28:46.543751 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.543701 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8"} err="failed to get container status \"f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8\": rpc error: code = NotFound desc = could not find container \"f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8\": container with ID starting with f94fd9bbfe6a31e7c0999a494708306894db5db792c0032233a11f85876cceb8 not found: ID does not exist" Jul 28 17:28:46.543751 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.543720 2554 scope.go:117] "RemoveContainer" containerID="0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e" Jul 28 17:28:46.544036 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:28:46.544020 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e\": container with ID starting with 0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e not found: ID does not exist" containerID="0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e" Jul 28 17:28:46.544088 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.544042 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e"} err="failed to get container status \"0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e\": rpc error: code = NotFound desc = could not find container \"0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e\": container with ID starting with 0b80064f13ecbfafdc67c73377b4f97437135b9eae2966ec83174277b041d03e not found: ID does not exist" Jul 28 17:28:46.544088 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.544058 2554 scope.go:117] "RemoveContainer" containerID="32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6" Jul 28 17:28:46.544300 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:28:46.544281 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6\": container with ID starting with 32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6 not found: ID does not exist" containerID="32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6" Jul 28 17:28:46.544347 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.544307 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6"} err="failed to get container status \"32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6\": rpc error: code = NotFound desc = could not find container \"32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6\": container with ID starting with 32bc3fc4e98dd3bea170ae4d81b52157ee51023c39d53ed78125b47395999ab6 not found: ID does not exist" Jul 28 17:28:46.544347 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:46.544311 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-8fhcb"] Jul 28 17:28:47.527201 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.527165 2554 generic.go:356] "Generic (PLEG): container finished" podID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerID="a827fe3367efe90323cca9f70055a476e184357b854f17022910c85bf58417cc" exitCode=0 Jul 28 17:28:47.527597 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.527241 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerDied","Data":"a827fe3367efe90323cca9f70055a476e184357b854f17022910c85bf58417cc"} Jul 28 17:28:47.669856 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.669826 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:28:47.736131 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736090 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls\") pod \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " Jul 28 17:28:47.736315 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736142 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgm2c\" (UniqueName: \"kubernetes.io/projected/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-api-access-bgm2c\") pod \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " Jul 28 17:28:47.736315 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736169 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-rbac-proxy-sar-config\") pod \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " Jul 28 17:28:47.736315 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736227 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-cabundle-cert\") pod \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " Jul 28 17:28:47.736315 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736271 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kserve-provision-location\") pod \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\" (UID: \"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff\") " Jul 28 17:28:47.736616 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736584 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" (UID: "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:28:47.736745 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736602 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" (UID: "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:28:47.736745 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.736701 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" (UID: "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:28:47.739113 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.739085 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-api-access-bgm2c" (OuterVolumeSpecName: "kube-api-access-bgm2c") pod "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" (UID: "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff"). InnerVolumeSpecName "kube-api-access-bgm2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:28:47.739113 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.739085 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" (UID: "4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:28:47.766755 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.766729 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:28:47.837716 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.837673 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-kube-rbac-proxy-sar-config\") pod \"149c3c47-5c42-4518-a006-4977e053d6f1\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " Jul 28 17:28:47.837969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.837757 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/149c3c47-5c42-4518-a006-4977e053d6f1-kserve-provision-location\") pod \"149c3c47-5c42-4518-a006-4977e053d6f1\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " Jul 28 17:28:47.837969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.837795 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-cabundle-cert\") pod \"149c3c47-5c42-4518-a006-4977e053d6f1\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " Jul 28 17:28:47.837969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.837829 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/149c3c47-5c42-4518-a006-4977e053d6f1-proxy-tls\") pod \"149c3c47-5c42-4518-a006-4977e053d6f1\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " Jul 28 17:28:47.837969 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.837916 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2t8p\" (UniqueName: \"kubernetes.io/projected/149c3c47-5c42-4518-a006-4977e053d6f1-kube-api-access-r2t8p\") pod \"149c3c47-5c42-4518-a006-4977e053d6f1\" (UID: \"149c3c47-5c42-4518-a006-4977e053d6f1\") " Jul 28 17:28:47.838202 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838162 2554 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-cabundle-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.838202 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838185 2554 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kserve-provision-location\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.838301 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838201 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-proxy-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.838301 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838218 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgm2c\" (UniqueName: \"kubernetes.io/projected/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-api-access-bgm2c\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.838301 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838231 2554 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.838301 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838161 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/149c3c47-5c42-4518-a006-4977e053d6f1-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "149c3c47-5c42-4518-a006-4977e053d6f1" (UID: "149c3c47-5c42-4518-a006-4977e053d6f1"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:28:47.838301 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838200 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "149c3c47-5c42-4518-a006-4977e053d6f1" (UID: "149c3c47-5c42-4518-a006-4977e053d6f1"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:28:47.838301 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.838195 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "149c3c47-5c42-4518-a006-4977e053d6f1" (UID: "149c3c47-5c42-4518-a006-4977e053d6f1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:28:47.840286 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.840256 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/149c3c47-5c42-4518-a006-4977e053d6f1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "149c3c47-5c42-4518-a006-4977e053d6f1" (UID: "149c3c47-5c42-4518-a006-4977e053d6f1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:28:47.840374 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.840317 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/149c3c47-5c42-4518-a006-4977e053d6f1-kube-api-access-r2t8p" (OuterVolumeSpecName: "kube-api-access-r2t8p") pod "149c3c47-5c42-4518-a006-4977e053d6f1" (UID: "149c3c47-5c42-4518-a006-4977e053d6f1"). InnerVolumeSpecName "kube-api-access-r2t8p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:28:47.939702 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.939586 2554 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/149c3c47-5c42-4518-a006-4977e053d6f1-kserve-provision-location\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.939702 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.939623 2554 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-cabundle-cert\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.939702 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.939633 2554 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/149c3c47-5c42-4518-a006-4977e053d6f1-proxy-tls\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.939702 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.939641 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2t8p\" (UniqueName: \"kubernetes.io/projected/149c3c47-5c42-4518-a006-4977e053d6f1-kube-api-access-r2t8p\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:47.939702 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:47.939650 2554 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/149c3c47-5c42-4518-a006-4977e053d6f1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 17:28:48.337703 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.337667 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" path="/var/lib/kubelet/pods/c37a984c-db24-4b00-aa53-82cfe0722356/volumes" Jul 28 17:28:48.534281 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.534245 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" Jul 28 17:28:48.534742 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.534242 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd" event={"ID":"4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff","Type":"ContainerDied","Data":"81587563b8f0d734f8310c7c16da083da99ac8684d9d76715ea7bcb9bc1d6aa0"} Jul 28 17:28:48.534742 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.534421 2554 scope.go:117] "RemoveContainer" containerID="022d73cebb7a2c5a61c7f139b42ceaac5e653dadedeae6ed94fee079060eadea" Jul 28 17:28:48.536291 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.536264 2554 generic.go:356] "Generic (PLEG): container finished" podID="149c3c47-5c42-4518-a006-4977e053d6f1" containerID="6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79" exitCode=0 Jul 28 17:28:48.536412 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.536328 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerDied","Data":"6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79"} Jul 28 17:28:48.536412 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.536354 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" Jul 28 17:28:48.536412 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.536378 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr" event={"ID":"149c3c47-5c42-4518-a006-4977e053d6f1","Type":"ContainerDied","Data":"49a43239ceb84baa9b5c6315ae0efea463801d92bdde3f35533942d8421f50d3"} Jul 28 17:28:48.543053 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.543038 2554 scope.go:117] "RemoveContainer" containerID="a827fe3367efe90323cca9f70055a476e184357b854f17022910c85bf58417cc" Jul 28 17:28:48.550643 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.550625 2554 scope.go:117] "RemoveContainer" containerID="f7352fe91927e6ac08bbbe34818780a78c79696cac72cab3510dd397c2df4bc3" Jul 28 17:28:48.557815 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.557796 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd"] Jul 28 17:28:48.558382 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.558364 2554 scope.go:117] "RemoveContainer" containerID="1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f" Jul 28 17:28:48.565559 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.565524 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7d44765f95-b55fd"] Jul 28 17:28:48.567347 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.567326 2554 scope.go:117] "RemoveContainer" containerID="6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79" Jul 28 17:28:48.576267 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.576241 2554 scope.go:117] "RemoveContainer" containerID="a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00" Jul 28 17:28:48.579659 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.579628 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr"] Jul 28 17:28:48.585389 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.585345 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6d55f5d589-c5gzr"] Jul 28 17:28:48.586692 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.586672 2554 scope.go:117] "RemoveContainer" containerID="1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f" Jul 28 17:28:48.587040 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:28:48.587018 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f\": container with ID starting with 1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f not found: ID does not exist" containerID="1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f" Jul 28 17:28:48.587095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.587052 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f"} err="failed to get container status \"1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f\": rpc error: code = NotFound desc = could not find container \"1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f\": container with ID starting with 1dd9f1edf1e00e9faffb54542ae5b99ad45a54e412f032f40bdfc0148508a79f not found: ID does not exist" Jul 28 17:28:48.587095 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.587074 2554 scope.go:117] "RemoveContainer" containerID="6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79" Jul 28 17:28:48.587375 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:28:48.587357 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79\": container with ID starting with 6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79 not found: ID does not exist" containerID="6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79" Jul 28 17:28:48.587420 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.587379 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79"} err="failed to get container status \"6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79\": rpc error: code = NotFound desc = could not find container \"6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79\": container with ID starting with 6a14ed1c657d916e3e7cc76cf47aa237f5978fbf02461da1b558e9a923d79f79 not found: ID does not exist" Jul 28 17:28:48.587420 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.587392 2554 scope.go:117] "RemoveContainer" containerID="a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00" Jul 28 17:28:48.587623 ip-10-0-129-206 kubenswrapper[2554]: E0728 17:28:48.587606 2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00\": container with ID starting with a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00 not found: ID does not exist" containerID="a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00" Jul 28 17:28:48.587665 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:48.587628 2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00"} err="failed to get container status \"a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00\": rpc error: code = NotFound desc = could not find container \"a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00\": container with ID starting with a6dc8d3e8746b54310dec2734cc5e87961e21cd63f5ce426b66d74556bc39a00 not found: ID does not exist" Jul 28 17:28:50.336819 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:50.336786 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" path="/var/lib/kubelet/pods/149c3c47-5c42-4518-a006-4977e053d6f1/volumes" Jul 28 17:28:50.337312 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:28:50.337298 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" path="/var/lib/kubelet/pods/4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff/volumes" Jul 28 17:29:41.898033 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:29:41.897999 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:29:42.185657 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:29:42.185556 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:32:01.126501 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:32:01.126461 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:33:06.044256 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:33:06.044204 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:33:06.106519 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:33:06.106477 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:33:36.274587 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:33:36.274549 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:34:43.496317 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:34:43.496269 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:34:43.558478 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:34:43.558439 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:00.602504 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:00.602389 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:00.676305 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:00.676266 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:00.742047 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:00.741995 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:00.808850 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:00.808807 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:00.876866 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:00.876769 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:01.005181 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:01.005120 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:01.153392 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:01.153268 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:56.606973 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:56.606927 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:56.690039 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:56.689986 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:56.758795 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:56.758751 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:56.878002 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:56.877908 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:56.976801 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:56.976764 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:37:57.206430 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:37:57.206322 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:00.805339 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:00.805258 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:00.905252 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:00.905202 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:10.887791 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:10.887700 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:11.005795 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:11.005756 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:11.100221 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:11.100143 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:11.220833 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:11.220745 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:11.347100 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:11.347063 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:11.477510 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:11.477419 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:56.836888 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:56.836842 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:38:58.315632 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:38:58.315596 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:06.924146 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:06.924064 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:07.030635 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:07.030598 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:07.105742 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:07.105684 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:07.181681 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:07.181572 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:07.266864 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:07.266793 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:07.832069 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:07.832004 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:21.127150 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:21.127107 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:39:21.310449 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:39:21.310413 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:40:07.159267 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:40:07.159142 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:40:07.430507 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:40:07.430406 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:41:48.664107 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:41:48.664072 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:41:56.794128 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:41:56.794084 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:47:36.662115 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:47:36.662055 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:47:38.409971 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:47:38.409915 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:17.775253 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:17.775208 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:21.706758 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:21.706698 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:21.803017 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:21.802962 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:21.901064 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:21.901020 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:21.966419 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:21.966319 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:22.069973 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:22.069868 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:22.347038 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:22.346986 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:36.086761 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:36.086708 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:36.186349 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:36.186306 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:46.183468 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:46.183334 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:46.273531 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:46.273487 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:46.351064 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:46.351018 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:46.467512 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:46.467412 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:46.594243 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:46.594190 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:48:46.881236 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:48:46.881194 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:49:31.978377 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:49:31.978306 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:49:32.758464 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:49:32.758423 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:49:56.462367 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:49:56.462301 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:49:56.562176 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:49:56.562122 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:50:15.824592 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:50:15.824459 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:50:15.886907 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:50:15.886833 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:51:17.499583 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:51:17.499538 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:52:58.917271 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:52:58.917231 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:54:46.987512 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:54:46.987426 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:57:50.603575 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:57:50.603494 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:58:11.129645 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:58:11.129606 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 17:58:12.698686 ip-10-0-129-206 kubenswrapper[2554]: I0728 17:58:12.698646 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:00:32.213217 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:00:32.213181 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:03:20.605397 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:03:20.605358 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:05:51.277446 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:51.277407 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:05:51.447326 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:51.447293 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:05:51.527997 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:51.527908 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:05:52.804811 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.804772 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hm8rd/must-gather-4g4rx"] Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805132 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kube-rbac-proxy" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805145 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kube-rbac-proxy" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805157 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="storage-initializer" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805163 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="storage-initializer" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805169 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805175 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805182 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kube-rbac-proxy" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805187 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kube-rbac-proxy" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805195 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="storage-initializer" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805201 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="storage-initializer" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805213 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="storage-initializer" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805218 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="storage-initializer" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805226 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" Jul 28 18:05:52.805227 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805231 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805254 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805261 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805270 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kube-rbac-proxy" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805275 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kube-rbac-proxy" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805368 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kserve-container" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805376 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kserve-container" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805382 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kube-rbac-proxy" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805387 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c37a984c-db24-4b00-aa53-82cfe0722356" containerName="kube-rbac-proxy" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805394 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae5b6aa-0fb0-4cfa-ba5c-838123a8d2ff" containerName="kserve-container" Jul 28 18:05:52.805731 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.805403 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="149c3c47-5c42-4518-a006-4977e053d6f1" containerName="kube-rbac-proxy" Jul 28 18:05:52.808398 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.808367 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:52.811508 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.811481 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hm8rd\"/\"kube-root-ca.crt\"" Jul 28 18:05:52.812968 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.812945 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hm8rd\"/\"default-dockercfg-g8mm7\"" Jul 28 18:05:52.813075 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.812950 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hm8rd\"/\"openshift-service-ca.crt\"" Jul 28 18:05:52.816960 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.816937 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hm8rd/must-gather-4g4rx"] Jul 28 18:05:52.937534 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.937511 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c1f71695-7f6e-4a7b-97f7-829b55df852f-must-gather-output\") pod \"must-gather-4g4rx\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:52.937640 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:52.937586 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhbzs\" (UniqueName: \"kubernetes.io/projected/c1f71695-7f6e-4a7b-97f7-829b55df852f-kube-api-access-xhbzs\") pod \"must-gather-4g4rx\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:53.037946 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.037917 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhbzs\" (UniqueName: \"kubernetes.io/projected/c1f71695-7f6e-4a7b-97f7-829b55df852f-kube-api-access-xhbzs\") pod \"must-gather-4g4rx\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:53.038076 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.037964 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c1f71695-7f6e-4a7b-97f7-829b55df852f-must-gather-output\") pod \"must-gather-4g4rx\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:53.038290 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.038270 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c1f71695-7f6e-4a7b-97f7-829b55df852f-must-gather-output\") pod \"must-gather-4g4rx\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:53.050178 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.050150 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhbzs\" (UniqueName: \"kubernetes.io/projected/c1f71695-7f6e-4a7b-97f7-829b55df852f-kube-api-access-xhbzs\") pod \"must-gather-4g4rx\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:53.128160 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.128108 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:05:53.213072 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.213040 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:05:53.260239 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.260219 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hm8rd/must-gather-4g4rx"] Jul 28 18:05:53.262046 ip-10-0-129-206 kubenswrapper[2554]: W0728 18:05:53.262018 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1f71695_7f6e_4a7b_97f7_829b55df852f.slice/crio-6f902c3bd2a5d9c7d2dadb29eb29572a5d3a6371438bc3d42eb6108339d2c1d5 WatchSource:0}: Error finding container 6f902c3bd2a5d9c7d2dadb29eb29572a5d3a6371438bc3d42eb6108339d2c1d5: Status 404 returned error can't find the container with id 6f902c3bd2a5d9c7d2dadb29eb29572a5d3a6371438bc3d42eb6108339d2c1d5 Jul 28 18:05:53.263761 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.263747 2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 18:05:53.308454 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.308431 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:05:53.962392 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:53.962359 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" event={"ID":"c1f71695-7f6e-4a7b-97f7-829b55df852f","Type":"ContainerStarted","Data":"6f902c3bd2a5d9c7d2dadb29eb29572a5d3a6371438bc3d42eb6108339d2c1d5"} Jul 28 18:05:59.991992 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:59.991952 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" event={"ID":"c1f71695-7f6e-4a7b-97f7-829b55df852f","Type":"ContainerStarted","Data":"c43276f19eedd524814f8e240f46a94d4370d67ab66fdd3bf86a96c2ba9a3127"} Jul 28 18:05:59.991992 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:05:59.991999 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" event={"ID":"c1f71695-7f6e-4a7b-97f7-829b55df852f","Type":"ContainerStarted","Data":"a3da7273173634f07bde0683eb87a94ffb76b5c19249df1bb66720f0f23fdcb8"} Jul 28 18:06:00.013002 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:00.012956 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" podStartSLOduration=2.079787146 podStartE2EDuration="8.012940748s" totalImagesPullingTime="5.933153602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 18:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:00.010152796 +0000 UTC m=+2918.228194259" watchObservedRunningTime="2026-07-28 18:06:00.012940748 +0000 UTC m=+2918.230982209" Jul 28 18:06:23.097599 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:23.097565 2554 generic.go:356] "Generic (PLEG): container finished" podID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerID="a3da7273173634f07bde0683eb87a94ffb76b5c19249df1bb66720f0f23fdcb8" exitCode=0 Jul 28 18:06:23.098024 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:23.097636 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" event={"ID":"c1f71695-7f6e-4a7b-97f7-829b55df852f","Type":"ContainerDied","Data":"a3da7273173634f07bde0683eb87a94ffb76b5c19249df1bb66720f0f23fdcb8"} Jul 28 18:06:23.098024 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:23.097923 2554 scope.go:117] "RemoveContainer" containerID="a3da7273173634f07bde0683eb87a94ffb76b5c19249df1bb66720f0f23fdcb8" Jul 28 18:06:23.460242 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:23.460159 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-hm8rd_must-gather-4g4rx_c1f71695-7f6e-4a7b-97f7-829b55df852f/gather/0.log" Jul 28 18:06:23.929456 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:23.929419 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:06:24.094817 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:24.094773 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:06:24.177447 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:24.177407 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:06:26.887795 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:26.887761 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xq4v4_fda79549-f59d-4702-b9a4-09416b782d4c/global-pull-secret-syncer/0.log" Jul 28 18:06:26.951901 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:26.951858 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" Jul 28 18:06:27.066627 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:27.066608 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-206.ec2.internal_fcb67e56a80fff0dbe2625d782081ff2/haproxy/0.log" Jul 28 18:06:28.906368 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:28.906333 2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-hm8rd/must-gather-4g4rx"] Jul 28 18:06:28.906842 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:28.906514 2554 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerName="copy" containerID="cri-o://c43276f19eedd524814f8e240f46a94d4370d67ab66fdd3bf86a96c2ba9a3127" gracePeriod=2 Jul 28 18:06:28.911916 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:28.911861 2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-hm8rd/must-gather-4g4rx"] Jul 28 18:06:29.124725 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.124699 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-hm8rd_must-gather-4g4rx_c1f71695-7f6e-4a7b-97f7-829b55df852f/copy/0.log" Jul 28 18:06:29.125116 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.125093 2554 generic.go:356] "Generic (PLEG): container finished" podID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerID="c43276f19eedd524814f8e240f46a94d4370d67ab66fdd3bf86a96c2ba9a3127" exitCode=143 Jul 28 18:06:29.125206 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.125175 2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f902c3bd2a5d9c7d2dadb29eb29572a5d3a6371438bc3d42eb6108339d2c1d5" Jul 28 18:06:29.131666 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.131652 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-hm8rd_must-gather-4g4rx_c1f71695-7f6e-4a7b-97f7-829b55df852f/copy/0.log" Jul 28 18:06:29.133285 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.133270 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:06:29.135915 ip-10-0-129-206 kubenswrapper[2554]: E0728 18:06:29.135894 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-4g4rx\" is forbidden: User \"system:node:ip-10-0-129-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-hm8rd\": no relationship found between node 'ip-10-0-129-206.ec2.internal' and this object" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:06:29.245499 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.245446 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c1f71695-7f6e-4a7b-97f7-829b55df852f-must-gather-output\") pod \"c1f71695-7f6e-4a7b-97f7-829b55df852f\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " Jul 28 18:06:29.245499 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.245475 2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhbzs\" (UniqueName: \"kubernetes.io/projected/c1f71695-7f6e-4a7b-97f7-829b55df852f-kube-api-access-xhbzs\") pod \"c1f71695-7f6e-4a7b-97f7-829b55df852f\" (UID: \"c1f71695-7f6e-4a7b-97f7-829b55df852f\") " Jul 28 18:06:29.246917 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.246893 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1f71695-7f6e-4a7b-97f7-829b55df852f-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "c1f71695-7f6e-4a7b-97f7-829b55df852f" (UID: "c1f71695-7f6e-4a7b-97f7-829b55df852f"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 18:06:29.247512 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.247481 2554 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1f71695-7f6e-4a7b-97f7-829b55df852f-kube-api-access-xhbzs" (OuterVolumeSpecName: "kube-api-access-xhbzs") pod "c1f71695-7f6e-4a7b-97f7-829b55df852f" (UID: "c1f71695-7f6e-4a7b-97f7-829b55df852f"). InnerVolumeSpecName "kube-api-access-xhbzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 18:06:29.346705 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.346684 2554 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/c1f71695-7f6e-4a7b-97f7-829b55df852f-must-gather-output\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 18:06:29.346705 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.346703 2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhbzs\" (UniqueName: \"kubernetes.io/projected/c1f71695-7f6e-4a7b-97f7-829b55df852f-kube-api-access-xhbzs\") on node \"ip-10-0-129-206.ec2.internal\" DevicePath \"\"" Jul 28 18:06:29.914482 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:29.914448 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:06:30.014206 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:30.014172 2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dv5bc_6cb9de26-57d7-4d29-8b2d-0c953175f120/konnectivity-agent/0.log" limit=1024 Jul 28 18:06:30.129090 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:30.129059 2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:06:30.131771 ip-10-0-129-206 kubenswrapper[2554]: E0728 18:06:30.131742 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-4g4rx\" is forbidden: User \"system:node:ip-10-0-129-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-hm8rd\": no relationship found between node 'ip-10-0-129-206.ec2.internal' and this object" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:06:30.139244 ip-10-0-129-206 kubenswrapper[2554]: E0728 18:06:30.139214 2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-4g4rx\" is forbidden: User \"system:node:ip-10-0-129-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-hm8rd\": no relationship found between node 'ip-10-0-129-206.ec2.internal' and this object" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" pod="openshift-must-gather-hm8rd/must-gather-4g4rx" Jul 28 18:06:30.336811 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:30.336785 2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" path="/var/lib/kubelet/pods/c1f71695-7f6e-4a7b-97f7-829b55df852f/volumes" Jul 28 18:06:30.951185 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:30.951155 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_004a4ee6-46fc-4fc2-b8a4-ece004bacb0a/alertmanager/0.log" Jul 28 18:06:30.981177 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:30.981154 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_004a4ee6-46fc-4fc2-b8a4-ece004bacb0a/config-reloader/0.log" Jul 28 18:06:31.013216 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.013195 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_004a4ee6-46fc-4fc2-b8a4-ece004bacb0a/kube-rbac-proxy-web/0.log" Jul 28 18:06:31.040226 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.040206 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_004a4ee6-46fc-4fc2-b8a4-ece004bacb0a/kube-rbac-proxy/0.log" Jul 28 18:06:31.069099 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.069083 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_004a4ee6-46fc-4fc2-b8a4-ece004bacb0a/kube-rbac-proxy-metric/0.log" Jul 28 18:06:31.095645 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.095621 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_004a4ee6-46fc-4fc2-b8a4-ece004bacb0a/prom-label-proxy/0.log" Jul 28 18:06:31.122887 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.122842 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_004a4ee6-46fc-4fc2-b8a4-ece004bacb0a/init-config-reloader/0.log" Jul 28 18:06:31.313908 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.313866 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5d5b5d9d67-5ddtt_432e2472-c9a3-479e-b840-f7202a4483cc/metrics-server/0.log" Jul 28 18:06:31.400403 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.400380 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67wqk_4b94e0cb-3eb3-4c81-aeba-0117020c9de0/node-exporter/0.log" Jul 28 18:06:31.429490 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.429471 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67wqk_4b94e0cb-3eb3-4c81-aeba-0117020c9de0/kube-rbac-proxy/0.log" Jul 28 18:06:31.463077 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.463061 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-67wqk_4b94e0cb-3eb3-4c81-aeba-0117020c9de0/init-textfile/0.log" Jul 28 18:06:31.821324 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.821287 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19a8081d-9516-4560-b605-4891c9a02af5/prometheus/0.log" Jul 28 18:06:31.848821 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.848797 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19a8081d-9516-4560-b605-4891c9a02af5/config-reloader/0.log" Jul 28 18:06:31.888200 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.888181 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19a8081d-9516-4560-b605-4891c9a02af5/thanos-sidecar/0.log" Jul 28 18:06:31.923031 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.923012 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19a8081d-9516-4560-b605-4891c9a02af5/kube-rbac-proxy-web/0.log" Jul 28 18:06:31.954428 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.954410 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19a8081d-9516-4560-b605-4891c9a02af5/kube-rbac-proxy/0.log" Jul 28 18:06:31.982380 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:31.982356 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19a8081d-9516-4560-b605-4891c9a02af5/kube-rbac-proxy-thanos/0.log" Jul 28 18:06:32.017736 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:32.017714 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_19a8081d-9516-4560-b605-4891c9a02af5/init-config-reloader/0.log" Jul 28 18:06:32.258589 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:32.258559 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55557df95f-8vrp5_68b16455-4ea2-4be1-bfef-56a619605594/thanos-query/0.log" Jul 28 18:06:32.283687 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:32.283660 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55557df95f-8vrp5_68b16455-4ea2-4be1-bfef-56a619605594/kube-rbac-proxy-web/0.log" Jul 28 18:06:32.313946 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:32.313922 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55557df95f-8vrp5_68b16455-4ea2-4be1-bfef-56a619605594/kube-rbac-proxy/0.log" Jul 28 18:06:32.340621 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:32.340600 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55557df95f-8vrp5_68b16455-4ea2-4be1-bfef-56a619605594/prom-label-proxy/0.log" Jul 28 18:06:32.370605 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:32.370588 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55557df95f-8vrp5_68b16455-4ea2-4be1-bfef-56a619605594/kube-rbac-proxy-rules/0.log" Jul 28 18:06:32.402030 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:32.402009 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55557df95f-8vrp5_68b16455-4ea2-4be1-bfef-56a619605594/kube-rbac-proxy-metrics/0.log" Jul 28 18:06:34.775781 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.775747 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-856748c4fb-skff4_5506192f-7633-4628-9559-b115d83bb42a/console/0.log" Jul 28 18:06:34.822290 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.822245 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-lh9t4_3a95e312-298d-4ad1-b495-8f8d98a7ccb4/download-server/0.log" Jul 28 18:06:34.863442 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.863415 2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8"] Jul 28 18:06:34.863723 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.863711 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerName="gather" Jul 28 18:06:34.863763 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.863724 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerName="gather" Jul 28 18:06:34.863763 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.863736 2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerName="copy" Jul 28 18:06:34.863763 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.863742 2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerName="copy" Jul 28 18:06:34.863858 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.863791 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerName="copy" Jul 28 18:06:34.863858 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.863802 2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f71695-7f6e-4a7b-97f7-829b55df852f" containerName="gather" Jul 28 18:06:34.866688 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.866669 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:34.869743 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.869725 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-92xdj\"/\"kube-root-ca.crt\"" Jul 28 18:06:34.869847 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.869729 2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-92xdj\"/\"default-dockercfg-5gwzn\"" Jul 28 18:06:34.871072 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.871053 2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-92xdj\"/\"openshift-service-ca.crt\"" Jul 28 18:06:34.876779 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.876761 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8"] Jul 28 18:06:34.988746 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.988721 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-podres\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:34.988869 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.988753 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-lib-modules\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:34.988869 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.988776 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-sys\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:34.988869 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.988793 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tph4f\" (UniqueName: \"kubernetes.io/projected/445756c8-33a7-4207-8797-222915d18862-kube-api-access-tph4f\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:34.989007 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:34.988890 2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-proc\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090269 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090211 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-podres\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090269 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090241 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-lib-modules\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090269 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090265 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-sys\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090471 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090289 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tph4f\" (UniqueName: \"kubernetes.io/projected/445756c8-33a7-4207-8797-222915d18862-kube-api-access-tph4f\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090471 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090334 2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-proc\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090471 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090365 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-sys\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090471 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090382 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-lib-modules\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090471 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090386 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-podres\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.090471 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.090411 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/445756c8-33a7-4207-8797-222915d18862-proc\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.099596 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.099573 2554 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tph4f\" (UniqueName: \"kubernetes.io/projected/445756c8-33a7-4207-8797-222915d18862-kube-api-access-tph4f\") pod \"perf-node-gather-daemonset-4gbw8\" (UID: \"445756c8-33a7-4207-8797-222915d18862\") " pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.176231 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.176201 2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:35.306294 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:35.306258 2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8"] Jul 28 18:06:35.312954 ip-10-0-129-206 kubenswrapper[2554]: W0728 18:06:35.312922 2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod445756c8_33a7_4207_8797_222915d18862.slice/crio-ab2a96841964169fd633fcd7178fdc3652400da4acefaaf5a55ebb13220cf2ea WatchSource:0}: Error finding container ab2a96841964169fd633fcd7178fdc3652400da4acefaaf5a55ebb13220cf2ea: Status 404 returned error can't find the container with id ab2a96841964169fd633fcd7178fdc3652400da4acefaaf5a55ebb13220cf2ea Jul 28 18:06:36.153047 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:36.153006 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" event={"ID":"445756c8-33a7-4207-8797-222915d18862","Type":"ContainerStarted","Data":"80b8bc628ea6df574cb4ea4337e3183bd0b2d5934212f4fb157d6ed3b4af3bd9"} Jul 28 18:06:36.153411 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:36.153069 2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" event={"ID":"445756c8-33a7-4207-8797-222915d18862","Type":"ContainerStarted","Data":"ab2a96841964169fd633fcd7178fdc3652400da4acefaaf5a55ebb13220cf2ea"} Jul 28 18:06:36.153411 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:36.153095 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:36.171833 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:36.171781 2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" podStartSLOduration=2.171764919 podStartE2EDuration="2.171764919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:06:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:06:36.171228474 +0000 UTC m=+2954.389269941" watchObservedRunningTime="2026-07-28 18:06:36.171764919 +0000 UTC m=+2954.389806382" Jul 28 18:06:36.424273 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:36.424155 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dzxzw_7eec1800-06ec-47c6-bad2-dab46839c8b2/dns/0.log" Jul 28 18:06:36.457739 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:36.457713 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dzxzw_7eec1800-06ec-47c6-bad2-dab46839c8b2/kube-rbac-proxy/0.log" Jul 28 18:06:36.653161 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:36.653127 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-m5mfn_014bd552-7857-4126-b50f-282e51e649bd/dns-node-resolver/0.log" Jul 28 18:06:37.215246 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:37.215218 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h7rmc_e32234d0-962e-463c-be7f-592397942fb4/node-ca/0.log" Jul 28 18:06:38.486799 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:38.486769 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4z9q8_52b67f5e-e899-4e00-b3ef-3529c95d7072/serve-healthcheck-canary/0.log" Jul 28 18:06:39.242345 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:39.242305 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rflm7_c0989e1e-1e45-401f-b97b-c8ec8500e786/kube-rbac-proxy/0.log" Jul 28 18:06:39.276210 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:39.276183 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rflm7_c0989e1e-1e45-401f-b97b-c8ec8500e786/exporter/0.log" Jul 28 18:06:39.313023 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:39.312992 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rflm7_c0989e1e-1e45-401f-b97b-c8ec8500e786/extractor/0.log" Jul 28 18:06:41.623920 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:41.623892 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-85d8b8ddfc-d97vr_cf7dc3f1-f8f5-4d3e-ade1-3d3767f23fe0/manager/0.log" Jul 28 18:06:41.842791 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:41.842761 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tvxnm_3b770d99-7056-4944-81bf-5c31df331e4a/s3-init/0.log" Jul 28 18:06:41.865038 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:41.865009 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-tvxnm_3b770d99-7056-4944-81bf-5c31df331e4a/download-hf-model/0.log" Jul 28 18:06:42.167567 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:42.167543 2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-92xdj/perf-node-gather-daemonset-4gbw8" Jul 28 18:06:46.523853 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:46.523820 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-q7fhd_858b4760-7ca2-47be-b8c5-fe46bdbbb932/migrator/0.log" Jul 28 18:06:46.552632 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:46.552601 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-q7fhd_858b4760-7ca2-47be-b8c5-fe46bdbbb932/graceful-termination/0.log" Jul 28 18:06:46.981594 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:46.981551 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc_ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3/kube-storage-version-migrator-operator/1.log" Jul 28 18:06:46.982796 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:46.982773 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-7xmtc_ce9c5b1d-7d82-40cc-a13f-3a78ade50ab3/kube-storage-version-migrator-operator/0.log" Jul 28 18:06:48.154638 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.154608 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tgzl_37e9f3f8-63cd-4635-9a06-d59209a1d68f/kube-multus-additional-cni-plugins/0.log" Jul 28 18:06:48.185076 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.185046 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tgzl_37e9f3f8-63cd-4635-9a06-d59209a1d68f/egress-router-binary-copy/0.log" Jul 28 18:06:48.211954 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.211932 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tgzl_37e9f3f8-63cd-4635-9a06-d59209a1d68f/cni-plugins/0.log" Jul 28 18:06:48.238624 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.238605 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tgzl_37e9f3f8-63cd-4635-9a06-d59209a1d68f/bond-cni-plugin/0.log" Jul 28 18:06:48.264281 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.264261 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tgzl_37e9f3f8-63cd-4635-9a06-d59209a1d68f/routeoverride-cni/0.log" Jul 28 18:06:48.291756 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.291737 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tgzl_37e9f3f8-63cd-4635-9a06-d59209a1d68f/whereabouts-cni-bincopy/0.log" Jul 28 18:06:48.319638 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.319620 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tgzl_37e9f3f8-63cd-4635-9a06-d59209a1d68f/whereabouts-cni/0.log" Jul 28 18:06:48.856291 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.856262 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-nz6p9_42991c9d-3418-4060-affe-ba9b2e9a9bae/kube-multus/0.log" Jul 28 18:06:48.910456 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.910427 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rbkjh_a7809890-59e0-45c7-9286-5f0fe38a4a64/network-metrics-daemon/0.log" Jul 28 18:06:48.934503 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:48.934475 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rbkjh_a7809890-59e0-45c7-9286-5f0fe38a4a64/kube-rbac-proxy/0.log" Jul 28 18:06:50.149676 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.149640 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/ovn-controller/0.log" Jul 28 18:06:50.183503 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.183471 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/ovn-acl-logging/0.log" Jul 28 18:06:50.215157 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.215126 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/kube-rbac-proxy-node/0.log" Jul 28 18:06:50.245988 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.245959 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 18:06:50.270087 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.270049 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/northd/0.log" Jul 28 18:06:50.298532 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.298506 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/nbdb/0.log" Jul 28 18:06:50.337438 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.337413 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/sbdb/0.log" Jul 28 18:06:50.416344 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:50.416263 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fjw6m_01d66eca-4c69-40a8-989e-0783e7affc09/ovnkube-controller/0.log" Jul 28 18:06:51.803060 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:51.803031 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-nb777_83970fbd-be09-483e-8d0c-162d170fe472/check-endpoints/0.log" Jul 28 18:06:51.867436 ip-10-0-129-206 kubenswrapper[2554]: I0728 18:06:51.867406 2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-fl4xc_8cba4b44-e7b9-43de-bdf8-5d4d936c6c9c/network-check-target-container/0.log"