Jul 28 16:17:55.543133 ip-10-0-132-37 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 28 16:17:55.543164 ip-10-0-132-37 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 28 16:17:55.543174 ip-10-0-132-37 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 28 16:17:55.543489 ip-10-0-132-37 systemd[1]: Failed to start Kubernetes Kubelet. Jul 28 16:18:05.198241 ip-10-0-132-37 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 28 16:18:05.198258 ip-10-0-132-37 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 946b98eeebe844afa950e7a81c82021b -- Jul 28 16:19:53.487425 ip-10-0-132-37 systemd[1]: Starting Kubernetes Kubelet... Jul 28 16:19:53.886184 ip-10-0-132-37 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:19:53.886184 ip-10-0-132-37 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 16:19:53.886184 ip-10-0-132-37 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:19:53.886184 ip-10-0-132-37 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 16:19:53.886184 ip-10-0-132-37 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 16:19:53.888321 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.888234 2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 16:19:53.892495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892481 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:19:53.892495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892494 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892498 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892502 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892505 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892510 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892514 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892517 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892520 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892523 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892527 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892530 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892536 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892539 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892542 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892545 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892547 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892550 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892552 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892555 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892558 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:19:53.892553 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892561 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892564 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892566 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892569 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892572 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892574 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892578 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892581 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892584 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892586 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892589 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892591 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892594 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892596 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892599 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892602 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892606 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892609 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892612 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892614 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:19:53.893025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892617 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892619 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892622 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892624 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892627 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892629 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892632 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892634 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892637 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892639 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892642 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892644 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892647 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892651 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892655 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892659 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892662 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892665 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892668 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:19:53.893567 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892670 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892673 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892675 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892678 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892680 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892683 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892685 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892688 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892691 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892693 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892696 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892699 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892701 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892704 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892706 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892709 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892711 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892714 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892716 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:19:53.894025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892719 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892721 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892724 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892746 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892750 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892753 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892757 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892761 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892764 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892767 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892770 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892772 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892776 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892778 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892781 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892783 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892786 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892789 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892791 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.892794 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:19:53.894492 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893487 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893494 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893497 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893499 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893502 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893505 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893508 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893511 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893514 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893516 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893519 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893522 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893524 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893527 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893529 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893532 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893537 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893540 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893542 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:19:53.894972 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893547 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893549 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893552 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893555 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893558 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893560 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893563 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893565 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893568 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893571 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893574 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893576 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893579 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893582 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893584 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893587 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893589 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893592 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893594 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893597 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:19:53.895459 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893600 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893604 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893609 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893611 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893615 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893618 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893620 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893623 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893625 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893628 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893630 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893633 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893635 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893638 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893640 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893643 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893645 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893648 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893650 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893653 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:19:53.895952 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893655 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893657 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893660 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893663 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893665 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893668 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893671 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893673 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893675 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893678 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893680 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893683 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893686 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893688 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893691 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893693 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893696 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893699 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893701 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893704 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:19:53.896495 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893707 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893710 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893712 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893715 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893717 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893720 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893725 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893728 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893731 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893735 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893737 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893740 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893744 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893746 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893749 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893753 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893756 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893759 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893761 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:19:53.896981 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.893764 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894113 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894118 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894121 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894123 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894126 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894129 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894132 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894134 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894139 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894157 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894161 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894164 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894168 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894170 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894173 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894176 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894178 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894181 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:19:53.897466 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894183 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894186 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894188 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894190 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894193 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894196 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894199 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894201 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894204 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894206 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894209 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894211 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894214 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894216 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894219 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894221 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894224 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894226 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894229 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894232 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:19:53.897927 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894235 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894238 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894240 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894243 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894245 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894249 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894253 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894256 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894258 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894261 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894264 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894267 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894269 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894272 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894274 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894277 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894279 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894282 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894285 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:19:53.898423 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894287 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894290 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894292 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894295 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894297 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894300 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894302 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894305 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894307 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894310 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894312 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894315 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894318 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894321 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894323 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894326 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894328 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894331 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894334 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894336 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894339 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:19:53.898948 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894341 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894344 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894347 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894349 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894352 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894354 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894357 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894359 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894364 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894367 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894369 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894372 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894374 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894377 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894379 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894382 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894385 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894387 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894390 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:19:53.899484 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894393 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.894396 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894462 2574 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894475 2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894483 2574 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894490 2574 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894496 2574 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894502 2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894507 2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894512 2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894515 2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894518 2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894522 2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894525 2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894528 2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894531 2574 flags.go:64] FLAG: --cgroup-root="" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894535 2574 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894538 2574 flags.go:64] FLAG: --client-ca-file="" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894540 2574 flags.go:64] FLAG: --cloud-provider="external" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894543 2574 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894548 2574 flags.go:64] FLAG: --cluster-domain="" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894552 2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894555 2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 16:19:53.899954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894559 2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894562 2574 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894565 2574 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894568 2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894571 2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894576 2574 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894579 2574 flags.go:64] FLAG: --contention-profiling="false" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894582 2574 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894584 2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894588 2574 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894590 2574 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894594 2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894597 2574 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894600 2574 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894603 2574 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894606 2574 flags.go:64] FLAG: --enable-server="true" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894609 2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894613 2574 flags.go:64] FLAG: --event-burst="100" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894616 2574 flags.go:64] FLAG: --event-qps="50" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894619 2574 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894622 2574 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894624 2574 flags.go:64] FLAG: --eviction-hard="" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894628 2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894631 2574 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894633 2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 16:19:53.900528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894636 2574 flags.go:64] FLAG: --eviction-soft="" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894639 2574 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894642 2574 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894645 2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894647 2574 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894652 2574 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894655 2574 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894658 2574 flags.go:64] FLAG: --feature-gates="" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894661 2574 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894664 2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894667 2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894670 2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894676 2574 flags.go:64] FLAG: --healthz-port="10248" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894679 2574 flags.go:64] FLAG: --help="false" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894681 2574 flags.go:64] FLAG: --hostname-override="ip-10-0-132-37.ec2.internal" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894684 2574 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894687 2574 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894690 2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894693 2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894697 2574 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894700 2574 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894702 2574 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894705 2574 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 16:19:53.901160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894708 2574 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894712 2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894715 2574 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894717 2574 flags.go:64] FLAG: --kube-reserved="" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894720 2574 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894723 2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894726 2574 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894729 2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894731 2574 flags.go:64] FLAG: --lock-file="" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894734 2574 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894737 2574 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894740 2574 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894745 2574 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894748 2574 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894752 2574 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894754 2574 flags.go:64] FLAG: --logging-format="text" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894757 2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894760 2574 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894763 2574 flags.go:64] FLAG: --manifest-url="" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894766 2574 flags.go:64] FLAG: --manifest-url-header="" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894770 2574 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894774 2574 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894778 2574 flags.go:64] FLAG: --max-pods="110" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894781 2574 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894784 2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 16:19:53.901794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894787 2574 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894790 2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894793 2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894796 2574 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894798 2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894805 2574 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894808 2574 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894811 2574 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894814 2574 flags.go:64] FLAG: --pod-cidr="" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894818 2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894823 2574 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894825 2574 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894828 2574 flags.go:64] FLAG: --pods-per-core="0" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894831 2574 flags.go:64] FLAG: --port="10250" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894834 2574 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894837 2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-060d1210a6829ca45" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894841 2574 flags.go:64] FLAG: --qos-reserved="" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894844 2574 flags.go:64] FLAG: --read-only-port="10255" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894846 2574 flags.go:64] FLAG: --register-node="true" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894849 2574 flags.go:64] FLAG: --register-with-taints="" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894853 2574 flags.go:64] FLAG: --registry-burst="10" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894857 2574 flags.go:64] FLAG: --registry-qps="5" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894860 2574 flags.go:64] FLAG: --reserved-cpus="" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894862 2574 flags.go:64] FLAG: --reserved-memory="" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894866 2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 16:19:53.902423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894869 2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894872 2574 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894875 2574 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894877 2574 flags.go:64] FLAG: --runonce="false" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894881 2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894884 2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894887 2574 flags.go:64] FLAG: --seccomp-default="false" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894890 2574 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894892 2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894895 2574 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894898 2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894901 2574 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894904 2574 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894906 2574 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894910 2574 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894912 2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894915 2574 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894919 2574 flags.go:64] FLAG: --system-cgroups="" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894921 2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894926 2574 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894929 2574 flags.go:64] FLAG: --tls-cert-file="" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894931 2574 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894936 2574 flags.go:64] FLAG: --tls-min-version="" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894938 2574 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894941 2574 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 16:19:53.903038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894944 2574 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894947 2574 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894949 2574 flags.go:64] FLAG: --v="2" Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894954 2574 flags.go:64] FLAG: --version="false" Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894958 2574 flags.go:64] FLAG: --vmodule="" Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894961 2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.894964 2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895071 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895075 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895078 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895081 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895085 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895088 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895091 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895093 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895096 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895099 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895102 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895105 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895107 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895110 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:19:53.903649 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895114 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895116 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895119 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895121 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895124 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895127 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895129 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895132 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895135 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895139 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895155 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895158 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895161 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895164 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895168 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895170 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895174 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895176 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895179 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:19:53.904171 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895181 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895184 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895186 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895190 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895193 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895196 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895198 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895201 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895203 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895206 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895209 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895211 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895214 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895216 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895219 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895221 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895224 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895226 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895230 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895233 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:19:53.904646 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895236 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895238 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895241 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895243 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895245 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895248 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895250 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895255 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895257 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895260 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895262 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895265 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895267 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895270 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895272 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895276 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895279 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895281 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895284 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:19:53.905329 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895287 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895291 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895294 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895297 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895300 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895303 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895305 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895308 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895310 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895313 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895316 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895318 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895321 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895324 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895327 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895330 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895332 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895335 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895337 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895340 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:19:53.906096 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895344 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895346 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895349 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895353 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895356 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895358 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.895361 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.896333 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.903185 2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.903202 2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.903873 2574 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.903881 2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903946 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903951 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:19:53.906765 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903954 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903957 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903961 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903964 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903966 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903969 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903972 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903974 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903977 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903980 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903982 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903985 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903988 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903990 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903994 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.903998 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904001 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904004 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904007 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:19:53.907118 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904009 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904012 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904015 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904017 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904020 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904023 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904025 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904028 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904037 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904045 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904047 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904050 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904053 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904057 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904061 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904064 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904066 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904069 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904072 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904074 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:19:53.907606 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904077 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904079 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904082 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904084 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904087 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904089 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904092 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904094 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904099 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904102 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904105 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904108 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904111 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904114 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904117 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904120 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904122 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904125 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904128 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904131 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:19:53.908133 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904134 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904158 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904161 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904164 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904167 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904169 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904172 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904175 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904178 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904180 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904183 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904186 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904188 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904191 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904194 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904197 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904199 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904201 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904204 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904206 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:19:53.908642 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904209 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904212 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904214 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904216 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904219 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904222 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904224 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904227 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904229 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904231 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904234 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904237 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904240 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904243 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904251 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904254 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904257 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:19:53.909136 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904259 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.904264 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904417 2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904422 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904425 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904428 2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904431 2574 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904433 2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904436 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904439 2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904441 2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904444 2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904446 2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904449 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904452 2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 16:19:53.909656 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904454 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904457 2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904459 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904462 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904465 2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904467 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904470 2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904472 2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904474 2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904477 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904479 2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904482 2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904485 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904488 2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904491 2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904498 2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904501 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904504 2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904506 2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 16:19:53.910025 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904508 2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904511 2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904513 2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904516 2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904520 2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904523 2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904526 2574 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904529 2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904531 2574 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904534 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904537 2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904539 2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904542 2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904544 2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904547 2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904549 2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904552 2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904555 2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904557 2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904559 2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 16:19:53.910500 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904562 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904564 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904573 2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904575 2574 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904578 2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904581 2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904585 2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904588 2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904591 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904598 2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904601 2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904604 2574 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904607 2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904609 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904612 2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904614 2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904617 2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904619 2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904621 2574 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904624 2574 feature_gate.go:328] unrecognized feature gate: Example Jul 28 16:19:53.910984 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904626 2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904629 2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904631 2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904634 2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904636 2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904638 2574 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904641 2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904644 2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904646 2574 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904649 2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904651 2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904654 2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904657 2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904659 2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904662 2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904666 2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904669 2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904671 2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904674 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 16:19:53.911474 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904677 2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904679 2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904682 2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904691 2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904693 2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904696 2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904698 2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:53.904701 2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.904706 2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.905111 2574 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.907549 2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.908778 2574 server.go:1019] "Starting client certificate rotation" Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.908873 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 16:19:53.911934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.908912 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 16:19:53.912271 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.911794 2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 16:19:53.915183 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.915169 2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 16:19:53.923731 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.923716 2574 log.go:25] "Validated CRI v1 runtime API" Jul 28 16:19:53.927342 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.927326 2574 log.go:25] "Validated CRI v1 image API" Jul 28 16:19:53.928183 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.928169 2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 16:19:53.930809 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.930787 2574 fs.go:135] Filesystem UUIDs: map[778f9528-8009-4961-a425-026a2b0857d1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cb29cb1b-7f13-44a3-aeae-46eec0b5bf1c:/dev/nvme0n1p3] Jul 28 16:19:53.930866 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.930809 2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 16:19:53.937569 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.937461 2574 manager.go:216] Machine: {Timestamp:2026-07-28 16:19:53.935066017 +0000 UTC m=+0.332170782 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3123209 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23063ac73355df964dd8ee01ccf00a SystemUUID:ec23063a-c733-55df-964d-d8ee01ccf00a BootID:946b98ee-ebe8-44af-a950-e7a81c82021b Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c1:eb:6f:85:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c1:eb:6f:85:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:76:6a:9a:6e:95 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 16:19:53.937569 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.937564 2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 16:19:53.937676 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.937651 2574 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 16:19:53.940787 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.940761 2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 16:19:53.940935 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.940790 2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-37.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 16:19:53.941328 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.941307 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 16:19:53.942082 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.942068 2574 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 16:19:53.942119 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.942085 2574 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 16:19:53.942119 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.942093 2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 16:19:53.942647 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.942637 2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 16:19:53.943413 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.943402 2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 16:19:53.945254 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.945243 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 28 16:19:53.945372 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.945363 2574 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 16:19:53.945402 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.945397 2574 kubelet.go:486] "Attempting to sync node with API server" Jul 28 16:19:53.945434 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.945418 2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 16:19:53.945465 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.945455 2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 16:19:53.945515 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.945507 2574 kubelet.go:398] "Adding apiserver pod source" Jul 28 16:19:53.945543 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.945521 2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 16:19:53.946054 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.946038 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 16:19:53.946054 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.946056 2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 16:19:53.948322 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.948307 2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 16:19:53.951325 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.951302 2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 16:19:53.951388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.951339 2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 16:19:53.953051 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953037 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 16:19:53.953096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953055 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 16:19:53.953096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953062 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 16:19:53.953096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953069 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 16:19:53.953096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953083 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 16:19:53.953096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953090 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 16:19:53.953096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953098 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 16:19:53.953259 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953103 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 16:19:53.953259 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953110 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 16:19:53.953259 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953115 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 16:19:53.953259 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953127 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 16:19:53.953259 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953132 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 16:19:53.953808 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953797 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 16:19:53.953837 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.953809 2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 16:19:53.957176 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.957162 2574 server.go:1268] "Started kubelet" Jul 28 16:19:53.957316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.957246 2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 16:19:53.957378 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.957337 2574 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 16:19:53.957429 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.957262 2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 16:19:53.957873 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.957852 2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-37.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 16:19:53.957873 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.957862 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 16:19:53.957995 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.957895 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 16:19:53.958050 ip-10-0-132-37 systemd[1]: Started Kubernetes Kubelet. Jul 28 16:19:53.959342 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.959323 2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 16:19:53.959410 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.959355 2574 server.go:310] "Adding debug handlers to kubelet server" Jul 28 16:19:53.962788 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.962757 2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 16:19:53.964473 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.963566 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-37.ec2.internal.18c680810717d519 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-37.ec2.internal,UID:ip-10-0-132-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-37.ec2.internal,},FirstTimestamp:2026-07-28 16:19:53.957119257 +0000 UTC m=+0.354224021,LastTimestamp:2026-07-28 16:19:53.957119257 +0000 UTC m=+0.354224021,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-37.ec2.internal,}" Jul 28 16:19:53.965688 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.965673 2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 16:19:53.965688 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.965682 2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 16:19:53.965804 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.965761 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:53.966463 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966195 2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 16:19:53.966463 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966204 2574 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 16:19:53.966463 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966377 2574 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 16:19:53.966647 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966522 2574 reconstruct.go:97] "Volume reconstruction finished" Jul 28 16:19:53.966647 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966549 2574 reconciler.go:29] "Reconciler: start to sync state" Jul 28 16:19:53.966748 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966691 2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 16:19:53.966748 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966701 2574 factory.go:55] Registering systemd factory Jul 28 16:19:53.966748 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966710 2574 factory.go:223] Registration of the systemd container factory successfully Jul 28 16:19:53.966899 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966888 2574 factory.go:153] Registering CRI-O factory Jul 28 16:19:53.966899 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966899 2574 factory.go:223] Registration of the crio container factory successfully Jul 28 16:19:53.966978 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966914 2574 factory.go:103] Registering Raw factory Jul 28 16:19:53.966978 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.966923 2574 manager.go:1196] Started watching for new ooms in manager Jul 28 16:19:53.967356 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.967343 2574 manager.go:349] Starting recovery of all containers Jul 28 16:19:53.968468 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.968444 2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 16:19:53.968549 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.968503 2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-37.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 16:19:53.969109 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.969087 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-54jst" Jul 28 16:19:53.976770 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.976746 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-54jst" Jul 28 16:19:53.980379 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.980360 2574 manager.go:354] Recovery completed Jul 28 16:19:53.984338 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.984324 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:19:53.986026 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.986006 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:19:53.986089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.986036 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:19:53.986089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.986046 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:19:53.986495 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.986482 2574 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 16:19:53.986495 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.986494 2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 16:19:53.986576 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.986508 2574 state_mem.go:36] "Initialized new in-memory state store" Jul 28 16:19:53.986576 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.986553 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:19:53.987138 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.987116 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:19:53.987213 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.987174 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:19:53.987213 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.987186 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:19:53.988000 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.987985 2574 policy_none.go:49] "None policy: Start" Jul 28 16:19:53.988000 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.988003 2574 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 16:19:53.988128 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.988015 2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 16:19:53.988597 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:53.988540 2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-37.ec2.internal.18c6808108d0dda7 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-37.ec2.internal,UID:ip-10-0-132-37.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-37.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-37.ec2.internal,},FirstTimestamp:2026-07-28 16:19:53.986022823 +0000 UTC m=+0.383127589,LastTimestamp:2026-07-28 16:19:53.986022823 +0000 UTC m=+0.383127589,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-37.ec2.internal,}" Jul 28 16:19:53.989255 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:53.989243 2574 policy_none.go:47] "Start" Jul 28 16:19:54.022834 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:54.022696 2574 manager.go:337] "Starting Device Plugin manager" Jul 28 16:19:54.022923 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:54.022856 2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 16:19:54.022923 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:54.022870 2574 server.go:87] "Starting device plugin registration server" Jul 28 16:19:54.023064 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:54.023052 2574 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 16:19:54.023114 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:54.023065 2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 16:19:54.023190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:54.023175 2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 16:19:54.023268 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:54.023256 2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 16:19:54.023268 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:54.023269 2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 16:19:54.023766 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:54.023751 2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 16:19:54.023830 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:54.023795 2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:55.662187 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.662158 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:19:55.663156 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.663057 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:19:55.663156 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.663086 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:19:55.663156 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.663096 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:19:55.663156 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.663120 2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.670912 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.670897 2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.670979 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.670919 2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-37.ec2.internal\": node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:55.683481 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.683437 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:55.693474 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.693443 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 16:19:55.694547 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.694523 2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 16:19:55.694627 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.694559 2574 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 16:19:55.694627 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.694579 2574 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 16:19:55.694627 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.694614 2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 16:19:55.698385 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.698370 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:19:55.783902 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.783882 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:55.795644 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.795624 2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal"] Jul 28 16:19:55.795695 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.795681 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:19:55.796450 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.796432 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:19:55.796533 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.796460 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:19:55.796533 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.796472 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:19:55.797689 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.797677 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:19:55.797816 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.797801 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.797855 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.797832 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:19:55.798344 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.798324 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:19:55.798344 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.798329 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:19:55.798465 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.798372 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:19:55.798465 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.798383 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:19:55.798465 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.798346 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:19:55.798465 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.798422 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:19:55.799544 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.799532 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.799597 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.799554 2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 16:19:55.800547 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.800530 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientMemory" Jul 28 16:19:55.800611 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.800563 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 16:19:55.800611 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.800579 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeHasSufficientPID" Jul 28 16:19:55.806961 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.806943 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eadeb8c45c8e8f79dc05f80b5906def8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal\" (UID: \"eadeb8c45c8e8f79dc05f80b5906def8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.807035 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.806968 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eadeb8c45c8e8f79dc05f80b5906def8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal\" (UID: \"eadeb8c45c8e8f79dc05f80b5906def8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.807035 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.806984 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/207f737ba8ca766d2b41e867763749e9-config\") pod \"kube-apiserver-proxy-ip-10-0-132-37.ec2.internal\" (UID: \"207f737ba8ca766d2b41e867763749e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.818280 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.818266 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-37.ec2.internal\" not found" node="ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.822454 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.822439 2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-37.ec2.internal\" not found" node="ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.884091 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.884074 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:55.907893 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.907866 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eadeb8c45c8e8f79dc05f80b5906def8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal\" (UID: \"eadeb8c45c8e8f79dc05f80b5906def8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.907893 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.907897 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eadeb8c45c8e8f79dc05f80b5906def8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal\" (UID: \"eadeb8c45c8e8f79dc05f80b5906def8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.908001 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.907915 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/207f737ba8ca766d2b41e867763749e9-config\") pod \"kube-apiserver-proxy-ip-10-0-132-37.ec2.internal\" (UID: \"207f737ba8ca766d2b41e867763749e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.908001 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.907940 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/207f737ba8ca766d2b41e867763749e9-config\") pod \"kube-apiserver-proxy-ip-10-0-132-37.ec2.internal\" (UID: \"207f737ba8ca766d2b41e867763749e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.908001 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.907954 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/eadeb8c45c8e8f79dc05f80b5906def8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal\" (UID: \"eadeb8c45c8e8f79dc05f80b5906def8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.908001 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:55.907962 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eadeb8c45c8e8f79dc05f80b5906def8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal\" (UID: \"eadeb8c45c8e8f79dc05f80b5906def8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:55.985181 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:55.985128 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.085625 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.085599 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.121125 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.121098 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:56.126274 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.126253 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" Jul 28 16:19:56.186660 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.186636 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.287173 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.287110 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.387655 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.387630 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.447228 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.447207 2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 16:19:56.447593 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.447347 2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 16:19:56.488705 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.488684 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.519696 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.519668 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 16:14:55 +0000 UTC" deadline="2028-04-21 18:17:20.326616602 +0000 UTC" Jul 28 16:19:56.519696 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.519693 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15193h57m23.806926307s" Jul 28 16:19:56.589000 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.588972 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.671412 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:56.671375 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod207f737ba8ca766d2b41e867763749e9.slice/crio-4920c55eac1aceea147cecaca1627e4eb92a89df41e068054976bf5e2d0e6f5b WatchSource:0}: Error finding container 4920c55eac1aceea147cecaca1627e4eb92a89df41e068054976bf5e2d0e6f5b: Status 404 returned error can't find the container with id 4920c55eac1aceea147cecaca1627e4eb92a89df41e068054976bf5e2d0e6f5b Jul 28 16:19:56.671603 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:56.671586 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeadeb8c45c8e8f79dc05f80b5906def8.slice/crio-6752fd35945bb992cbf0a4821ad48c419f5452f50f7aa25e6ce3a936d5670723 WatchSource:0}: Error finding container 6752fd35945bb992cbf0a4821ad48c419f5452f50f7aa25e6ce3a936d5670723: Status 404 returned error can't find the container with id 6752fd35945bb992cbf0a4821ad48c419f5452f50f7aa25e6ce3a936d5670723 Jul 28 16:19:56.675070 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.675052 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:19:56.689480 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.689463 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.697378 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.697332 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" event={"ID":"eadeb8c45c8e8f79dc05f80b5906def8","Type":"ContainerStarted","Data":"6752fd35945bb992cbf0a4821ad48c419f5452f50f7aa25e6ce3a936d5670723"} Jul 28 16:19:56.698298 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.698272 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" event={"ID":"207f737ba8ca766d2b41e867763749e9","Type":"ContainerStarted","Data":"4920c55eac1aceea147cecaca1627e4eb92a89df41e068054976bf5e2d0e6f5b"} Jul 28 16:19:56.700495 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.700479 2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:19:56.789686 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:56.789664 2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-37.ec2.internal\" not found" Jul 28 16:19:56.797104 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.797059 2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:19:56.805080 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.805065 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" Jul 28 16:19:56.817891 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.817875 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 16:19:56.819660 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.819648 2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" Jul 28 16:19:56.828461 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.828448 2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 16:19:56.895516 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:56.895491 2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:19:57.484893 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.484854 2574 apiserver.go:52] "Watching apiserver" Jul 28 16:19:57.496145 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.496116 2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 16:19:57.496639 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.496617 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-bqh6d","kube-system/konnectivity-agent-jx5jf","openshift-cluster-node-tuning-operator/tuned-nk8pj","openshift-image-registry/node-ca-pdvc9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal","openshift-multus/network-metrics-daemon-mgt4v","openshift-network-diagnostics/network-check-target-7hxvb","openshift-network-operator/iptables-alerter-jrbqd","kube-system/global-pull-secret-syncer-4wfgl","kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc","openshift-dns/node-resolver-rhfmx","openshift-multus/multus-additional-cni-plugins-8hjhz","openshift-multus/multus-tbkgq"] Jul 28 16:19:57.499129 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.499100 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:19:57.499238 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.499189 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:19:57.501188 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.501164 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.503506 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.503483 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.504644 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.504587 2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 16:19:57.505665 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.505644 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.506201 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.506129 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 16:19:57.506201 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.506156 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 16:19:57.506201 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.506175 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 16:19:57.506405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.506170 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 16:19:57.506405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.506184 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wj5md\"" Jul 28 16:19:57.506405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.506331 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:19:57.506657 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.506637 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g6ptl\"" Jul 28 16:19:57.508158 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.507920 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.508431 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.508413 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 16:19:57.508989 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.508967 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 16:19:57.509369 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.509352 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lxhmz\"" Jul 28 16:19:57.509962 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.509637 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 16:19:57.511287 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.510630 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 16:19:57.511287 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.510914 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 16:19:57.511287 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.511142 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 16:19:57.511287 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.511182 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l45qj\"" Jul 28 16:19:57.511287 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.511224 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 16:19:57.511917 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.511672 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:57.512335 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.512306 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:19:57.514729 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.514692 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.514832 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.514809 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:19:57.516621 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.516600 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysctl-conf\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.516816 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.516789 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-systemd\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.516919 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.516897 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.517237 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517204 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-run\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517237 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517242 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881f0dd5-06fb-4516-a54e-0f11567fcf93-tmp\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517266 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:19:57.517405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517288 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-sys-fs\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.517405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517308 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-sys\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517357 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-tuned\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517405 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517390 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-kubernetes\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517419 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-registration-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.517638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517444 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-device-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.517638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517467 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.517638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517491 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysctl-d\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517527 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-lib-modules\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517574 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-var-lib-kubelet\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517609 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-socket-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517638 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27lcq\" (UniqueName: \"kubernetes.io/projected/10b582ed-07c0-4cf7-9a9c-60424d33558c-kube-api-access-27lcq\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-host\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nlp\" (UniqueName: \"kubernetes.io/projected/881f0dd5-06fb-4516-a54e-0f11567fcf93-kube-api-access-g9nlp\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517730 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2db18f9-9353-4501-babc-b9f1e28ab745-host\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517776 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2db18f9-9353-4501-babc-b9f1e28ab745-serviceca\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517809 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g99kz\" (UniqueName: \"kubernetes.io/projected/d2db18f9-9353-4501-babc-b9f1e28ab745-kube-api-access-g99kz\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517839 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517862 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-modprobe-d\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.517985 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.517883 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysconfig\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.519172 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.519154 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.520020 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.520002 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 16:19:57.520131 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.520094 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-469jb\"" Jul 28 16:19:57.520131 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.520116 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 16:19:57.520251 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.520189 2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 16:19:57.521511 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.521186 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.521650 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.521632 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 16:19:57.521937 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.521918 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gkr2c\"" Jul 28 16:19:57.522045 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.521979 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 16:19:57.523904 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.523879 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.523999 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.523969 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 16:19:57.524173 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.524154 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 16:19:57.524384 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.524369 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 16:19:57.524551 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.524532 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pjrq4\"" Jul 28 16:19:57.526339 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.525930 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.527960 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.527941 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 16:19:57.528160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528143 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 16:19:57.528160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528154 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2dx7w\"" Jul 28 16:19:57.528351 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528336 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 16:19:57.528404 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528376 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 16:19:57.528567 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528553 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 16:19:57.528610 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528575 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 16:19:57.528877 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528859 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 16:19:57.528939 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528888 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 16:19:57.528939 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.528892 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zwzj7\"" Jul 28 16:19:57.552920 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.552854 2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-87x6f" Jul 28 16:19:57.561416 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.561092 2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-87x6f" Jul 28 16:19:57.573793 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.573777 2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 16:19:57.606576 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.606550 2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 16:19:57.618507 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618452 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g99kz\" (UniqueName: \"kubernetes.io/projected/d2db18f9-9353-4501-babc-b9f1e28ab745-kube-api-access-g99kz\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.618639 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618542 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ab0cb4d-8319-49fc-8031-7e2b24124055-dbus\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.618639 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618581 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e678a606-ebd7-4932-997b-12a94ef7f712-cni-binary-copy\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.618639 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.618824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618652 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-cni-bin\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.618824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618680 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-env-overrides\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.618824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618703 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/077326cd-7c10-4e5c-9c73-51637330d2c9-iptables-alerter-script\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.618824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618751 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw59g\" (UniqueName: \"kubernetes.io/projected/40b096ac-3f93-4ef8-a116-b3599aa57561-kube-api-access-nw59g\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.618824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618789 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-sys-fs\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.618824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618820 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-var-lib-kubelet\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.619089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618851 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-host\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.619089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618925 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-k8s-cni-cncf-io\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.619089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618956 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-systemd-units\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.619089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.618964 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-sys-fs\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.619089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619051 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-host\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.619260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619107 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.619260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619152 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:57.619260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-socket-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.619260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619230 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nlp\" (UniqueName: \"kubernetes.io/projected/881f0dd5-06fb-4516-a54e-0f11567fcf93-kube-api-access-g9nlp\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619487 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2db18f9-9353-4501-babc-b9f1e28ab745-host\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619541 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619591 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d2db18f9-9353-4501-babc-b9f1e28ab745-host\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619641 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-cnibin\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619673 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-slash\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619738 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-cni-netd\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619773 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q9gf\" (UniqueName: \"kubernetes.io/projected/0216df9b-1601-4dfc-b823-c0d00e69ee69-kube-api-access-6q9gf\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619803 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-socket-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619808 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-os-release\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619858 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-var-lib-kubelet\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619907 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-modprobe-d\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.619940 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysctl-conf\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-systemd\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620151 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovnkube-script-lib\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620229 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-modprobe-d\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620286 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:19:57.621607 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620313 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-sys\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620347 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90c74a1a-35fb-4b6f-b6b8-ab339710936a-konnectivity-ca\") pod \"konnectivity-agent-jx5jf\" (UID: \"90c74a1a-35fb-4b6f-b6b8-ab339710936a\") " pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620353 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-systemd\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620522 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-sys\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620466 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-cni-bin\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620621 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysctl-conf\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.620682 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-etc-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.621666 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-log-socket\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.621705 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovn-node-metrics-cert\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.621756 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-device-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.621816 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-systemd\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.621858 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.621887 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-device-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.621921 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27lcq\" (UniqueName: \"kubernetes.io/projected/10b582ed-07c0-4cf7-9a9c-60424d33558c-kube-api-access-27lcq\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622078 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-hostroot\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622143 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-system-cni-dir\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622177 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-kubelet\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.622483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622235 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-run-ovn-kubernetes\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881f0dd5-06fb-4516-a54e-0f11567fcf93-tmp\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622307 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e678a606-ebd7-4932-997b-12a94ef7f712-multus-daemon-config\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622429 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-os-release\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622459 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-multus-certs\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622504 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-run-netns\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622543 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-ovn\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622576 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-node-log\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622616 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovnkube-config\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622673 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27acf1b0-1531-400f-8d62-fd525316311d-tmp-dir\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622759 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg47t\" (UniqueName: \"kubernetes.io/projected/7600bbc2-65b3-4c9f-87aa-053bf079816c-kube-api-access-rg47t\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622866 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysctl-d\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622811 2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.622914 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2db18f9-9353-4501-babc-b9f1e28ab745-serviceca\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.623032 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-cni-multus\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.623072 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysctl-d\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.623075 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm8hc\" (UniqueName: \"kubernetes.io/projected/e678a606-ebd7-4932-997b-12a94ef7f712-kube-api-access-mm8hc\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.623249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.623146 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-cnibin\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624227 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624522 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-cni-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624547 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624565 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d2db18f9-9353-4501-babc-b9f1e28ab745-serviceca\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624576 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624601 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysconfig\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624623 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-run\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624643 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ab0cb4d-8319-49fc-8031-7e2b24124055-kubelet-config\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4xjv\" (UniqueName: \"kubernetes.io/projected/077326cd-7c10-4e5c-9c73-51637330d2c9-kube-api-access-g4xjv\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624687 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-system-cni-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624722 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-socket-dir-parent\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624682 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624745 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-conf-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624803 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-tuned\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624810 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-sysconfig\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624820 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90c74a1a-35fb-4b6f-b6b8-ab339710936a-agent-certs\") pod \"konnectivity-agent-jx5jf\" (UID: \"90c74a1a-35fb-4b6f-b6b8-ab339710936a\") " pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.624965 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624835 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/077326cd-7c10-4e5c-9c73-51637330d2c9-host-slash\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624856 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-netns\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-kubelet\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624902 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-var-lib-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624878 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-run\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624925 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27acf1b0-1531-400f-8d62-fd525316311d-hosts-file\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624947 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-kubernetes\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624962 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-registration-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624976 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.624992 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-lib-modules\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.625007 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-etc-kubernetes\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.625022 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.625036 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxtbh\" (UniqueName: \"kubernetes.io/projected/27acf1b0-1531-400f-8d62-fd525316311d-kube-api-access-pxtbh\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.625035 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-kubernetes\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.625053 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-etc-selinux\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.625145 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/10b582ed-07c0-4cf7-9a9c-60424d33558c-registration-dir\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.625903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.625187 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/881f0dd5-06fb-4516-a54e-0f11567fcf93-lib-modules\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.627229 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.626941 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:19:57.627229 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.626947 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/881f0dd5-06fb-4516-a54e-0f11567fcf93-tmp\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.627229 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.626964 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:19:57.627229 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.626977 2574 projected.go:196] Error preparing data for projected volume kube-api-access-lgjw5 for pod openshift-network-diagnostics/network-check-target-7hxvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:19:57.627229 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.627049 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5 podName:988926e8-4287-4223-a8b2-8f7d813fedfc nodeName:}" failed. No retries permitted until 2026-07-28 16:19:58.12702946 +0000 UTC m=+2.985569203 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lgjw5" (UniqueName: "kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5") pod "network-check-target-7hxvb" (UID: "988926e8-4287-4223-a8b2-8f7d813fedfc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:19:57.627535 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.627246 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/881f0dd5-06fb-4516-a54e-0f11567fcf93-etc-tuned\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.629367 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.629341 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g99kz\" (UniqueName: \"kubernetes.io/projected/d2db18f9-9353-4501-babc-b9f1e28ab745-kube-api-access-g99kz\") pod \"node-ca-pdvc9\" (UID: \"d2db18f9-9353-4501-babc-b9f1e28ab745\") " pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.630002 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.629979 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27lcq\" (UniqueName: \"kubernetes.io/projected/10b582ed-07c0-4cf7-9a9c-60424d33558c-kube-api-access-27lcq\") pod \"aws-ebs-csi-driver-node-6jjdc\" (UID: \"10b582ed-07c0-4cf7-9a9c-60424d33558c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.631830 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.631807 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nlp\" (UniqueName: \"kubernetes.io/projected/881f0dd5-06fb-4516-a54e-0f11567fcf93-kube-api-access-g9nlp\") pod \"tuned-nk8pj\" (UID: \"881f0dd5-06fb-4516-a54e-0f11567fcf93\") " pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.726064 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726029 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-systemd\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726064 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726069 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726096 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-hostroot\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726119 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-system-cni-dir\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726130 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-systemd\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726141 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-kubelet\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726144 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726169 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-run-ovn-kubernetes\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726180 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-hostroot\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726194 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e678a606-ebd7-4932-997b-12a94ef7f712-multus-daemon-config\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726202 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-system-cni-dir\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726219 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-os-release\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726237 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-run-ovn-kubernetes\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-multus-certs\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726275 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-run-netns\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726294 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-ovn\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.726301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726310 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-node-log\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726325 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovnkube-config\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726340 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27acf1b0-1531-400f-8d62-fd525316311d-tmp-dir\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726381 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-run-netns\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726220 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-kubelet\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726294 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-multus-certs\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726380 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg47t\" (UniqueName: \"kubernetes.io/projected/7600bbc2-65b3-4c9f-87aa-053bf079816c-kube-api-access-rg47t\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726449 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-cni-multus\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726483 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm8hc\" (UniqueName: \"kubernetes.io/projected/e678a606-ebd7-4932-997b-12a94ef7f712-kube-api-access-mm8hc\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726510 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-cnibin\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726530 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-run-ovn\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726536 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726587 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-cni-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726751 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-cni-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726799 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-node-log\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726349 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-os-release\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726880 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e678a606-ebd7-4932-997b-12a94ef7f712-multus-daemon-config\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726935 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-cnibin\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.727038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726939 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-cni-multus\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.726983 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727016 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ab0cb4d-8319-49fc-8031-7e2b24124055-kubelet-config\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727043 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4xjv\" (UniqueName: \"kubernetes.io/projected/077326cd-7c10-4e5c-9c73-51637330d2c9-kube-api-access-g4xjv\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727068 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-system-cni-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727092 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-socket-dir-parent\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727117 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-conf-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727135 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ab0cb4d-8319-49fc-8031-7e2b24124055-kubelet-config\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727142 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90c74a1a-35fb-4b6f-b6b8-ab339710936a-agent-certs\") pod \"konnectivity-agent-jx5jf\" (UID: \"90c74a1a-35fb-4b6f-b6b8-ab339710936a\") " pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727173 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/077326cd-7c10-4e5c-9c73-51637330d2c9-host-slash\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727198 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-netns\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovnkube-config\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727241 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-kubelet\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727265 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-var-lib-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27acf1b0-1531-400f-8d62-fd525316311d-hosts-file\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727318 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-etc-kubernetes\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727343 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.727955 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727368 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxtbh\" (UniqueName: \"kubernetes.io/projected/27acf1b0-1531-400f-8d62-fd525316311d-kube-api-access-pxtbh\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727379 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-system-cni-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727400 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ab0cb4d-8319-49fc-8031-7e2b24124055-dbus\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727426 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e678a606-ebd7-4932-997b-12a94ef7f712-cni-binary-copy\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727431 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-socket-dir-parent\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727089 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727468 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727485 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-multus-conf-dir\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727497 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-cni-bin\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-env-overrides\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727530 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-etc-kubernetes\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727549 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/077326cd-7c10-4e5c-9c73-51637330d2c9-iptables-alerter-script\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727552 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/27acf1b0-1531-400f-8d62-fd525316311d-tmp-dir\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727563 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/077326cd-7c10-4e5c-9c73-51637330d2c9-host-slash\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727575 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw59g\" (UniqueName: \"kubernetes.io/projected/40b096ac-3f93-4ef8-a116-b3599aa57561-kube-api-access-nw59g\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727556 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ab0cb4d-8319-49fc-8031-7e2b24124055-dbus\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727455 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-cni-binary-copy\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.728763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727618 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-k8s-cni-cncf-io\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727623 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-netns\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727641 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-systemd-units\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727668 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727735 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727740 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727780 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727808 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-cnibin\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727833 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-slash\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727855 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-cni-netd\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727873 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q9gf\" (UniqueName: \"kubernetes.io/projected/0216df9b-1601-4dfc-b823-c0d00e69ee69-kube-api-access-6q9gf\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727884 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727889 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-os-release\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.727835 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727938 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/27acf1b0-1531-400f-8d62-fd525316311d-hosts-file\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727939 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-systemd-units\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.727961 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs podName:7600bbc2-65b3-4c9f-87aa-053bf079816c nodeName:}" failed. No retries permitted until 2026-07-28 16:19:58.227947457 +0000 UTC m=+3.086487177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs") pod "network-metrics-daemon-mgt4v" (UID: "7600bbc2-65b3-4c9f-87aa-053bf079816c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:19:57.729582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727969 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-kubelet\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.727985 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-env-overrides\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.728004 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728034 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-var-lib-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728049 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-cnibin\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:57.728106 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret podName:9ab0cb4d-8319-49fc-8031-7e2b24124055 nodeName:}" failed. No retries permitted until 2026-07-28 16:19:58.22809094 +0000 UTC m=+3.086630662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret") pod "global-pull-secret-syncer-4wfgl" (UID: "9ab0cb4d-8319-49fc-8031-7e2b24124055") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728115 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e678a606-ebd7-4932-997b-12a94ef7f712-cni-binary-copy\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728191 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-cni-bin\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728255 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40b096ac-3f93-4ef8-a116-b3599aa57561-os-release\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovnkube-script-lib\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728300 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-run-k8s-cni-cncf-io\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728320 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90c74a1a-35fb-4b6f-b6b8-ab339710936a-konnectivity-ca\") pod \"konnectivity-agent-jx5jf\" (UID: \"90c74a1a-35fb-4b6f-b6b8-ab339710936a\") " pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728347 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-cni-bin\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728371 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-etc-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728400 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-slash\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728408 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-host-cni-netd\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728414 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-etc-openvswitch\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.730343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728457 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e678a606-ebd7-4932-997b-12a94ef7f712-host-var-lib-cni-bin\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728461 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-log-socket\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728478 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/40b096ac-3f93-4ef8-a116-b3599aa57561-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728494 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0216df9b-1601-4dfc-b823-c0d00e69ee69-log-socket\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovn-node-metrics-cert\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.728550 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/077326cd-7c10-4e5c-9c73-51637330d2c9-iptables-alerter-script\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.729053 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovnkube-script-lib\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.729517 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/90c74a1a-35fb-4b6f-b6b8-ab339710936a-konnectivity-ca\") pod \"konnectivity-agent-jx5jf\" (UID: \"90c74a1a-35fb-4b6f-b6b8-ab339710936a\") " pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.730467 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/90c74a1a-35fb-4b6f-b6b8-ab339710936a-agent-certs\") pod \"konnectivity-agent-jx5jf\" (UID: \"90c74a1a-35fb-4b6f-b6b8-ab339710936a\") " pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.731182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.730962 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0216df9b-1601-4dfc-b823-c0d00e69ee69-ovn-node-metrics-cert\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.734760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.734740 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm8hc\" (UniqueName: \"kubernetes.io/projected/e678a606-ebd7-4932-997b-12a94ef7f712-kube-api-access-mm8hc\") pod \"multus-tbkgq\" (UID: \"e678a606-ebd7-4932-997b-12a94ef7f712\") " pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.735472 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.735398 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg47t\" (UniqueName: \"kubernetes.io/projected/7600bbc2-65b3-4c9f-87aa-053bf079816c-kube-api-access-rg47t\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:57.736379 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.736336 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4xjv\" (UniqueName: \"kubernetes.io/projected/077326cd-7c10-4e5c-9c73-51637330d2c9-kube-api-access-g4xjv\") pod \"iptables-alerter-jrbqd\" (UID: \"077326cd-7c10-4e5c-9c73-51637330d2c9\") " pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.737389 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.737361 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxtbh\" (UniqueName: \"kubernetes.io/projected/27acf1b0-1531-400f-8d62-fd525316311d-kube-api-access-pxtbh\") pod \"node-resolver-rhfmx\" (UID: \"27acf1b0-1531-400f-8d62-fd525316311d\") " pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:57.741569 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.741549 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q9gf\" (UniqueName: \"kubernetes.io/projected/0216df9b-1601-4dfc-b823-c0d00e69ee69-kube-api-access-6q9gf\") pod \"ovnkube-node-bqh6d\" (UID: \"0216df9b-1601-4dfc-b823-c0d00e69ee69\") " pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.743464 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.743443 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw59g\" (UniqueName: \"kubernetes.io/projected/40b096ac-3f93-4ef8-a116-b3599aa57561-kube-api-access-nw59g\") pod \"multus-additional-cni-plugins-8hjhz\" (UID: \"40b096ac-3f93-4ef8-a116-b3599aa57561\") " pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.816395 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.816365 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" Jul 28 16:19:57.824198 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.824181 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" Jul 28 16:19:57.838930 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.838899 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pdvc9" Jul 28 16:19:57.844583 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.844554 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-tbkgq" Jul 28 16:19:57.852184 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.852165 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" Jul 28 16:19:57.859756 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.859738 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:19:57.868252 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.868233 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jrbqd" Jul 28 16:19:57.876847 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.876831 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:19:57.882243 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:57.882225 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rhfmx" Jul 28 16:19:58.130905 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.130824 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:19:58.131052 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.130980 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:19:58.131052 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.131005 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:19:58.131052 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.131020 2574 projected.go:196] Error preparing data for projected volume kube-api-access-lgjw5 for pod openshift-network-diagnostics/network-check-target-7hxvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:19:58.131184 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.131083 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5 podName:988926e8-4287-4223-a8b2-8f7d813fedfc nodeName:}" failed. No retries permitted until 2026-07-28 16:19:59.131064218 +0000 UTC m=+3.989603939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgjw5" (UniqueName: "kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5") pod "network-check-target-7hxvb" (UID: "988926e8-4287-4223-a8b2-8f7d813fedfc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:19:58.231122 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.231093 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:58.231122 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.231122 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:58.231311 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.231237 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:19:58.231311 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.231279 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:19:58.231311 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.231299 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs podName:7600bbc2-65b3-4c9f-87aa-053bf079816c nodeName:}" failed. No retries permitted until 2026-07-28 16:19:59.231282637 +0000 UTC m=+4.089822372 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs") pod "network-metrics-daemon-mgt4v" (UID: "7600bbc2-65b3-4c9f-87aa-053bf079816c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:19:58.231422 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.231316 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret podName:9ab0cb4d-8319-49fc-8031-7e2b24124055 nodeName:}" failed. No retries permitted until 2026-07-28 16:19:59.231304223 +0000 UTC m=+4.089843944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret") pod "global-pull-secret-syncer-4wfgl" (UID: "9ab0cb4d-8319-49fc-8031-7e2b24124055") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:19:58.564624 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.564553 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 16:14:57 +0000 UTC" deadline="2028-02-09 16:14:01.755059296 +0000 UTC" Jul 28 16:19:58.564624 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.564583 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13463h54m3.19047988s" Jul 28 16:19:58.607243 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:58.606953 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40b096ac_3f93_4ef8_a116_b3599aa57561.slice/crio-8ed3b09e0b1466c90353ed20f4b12cb6acbc2be348adc49c2501965595348c9b WatchSource:0}: Error finding container 8ed3b09e0b1466c90353ed20f4b12cb6acbc2be348adc49c2501965595348c9b: Status 404 returned error can't find the container with id 8ed3b09e0b1466c90353ed20f4b12cb6acbc2be348adc49c2501965595348c9b Jul 28 16:19:58.608269 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:58.608234 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod881f0dd5_06fb_4516_a54e_0f11567fcf93.slice/crio-62090f85c3336e5a22c648ae691fae255ccb461fc41e9437b002ab4aa51888d7 WatchSource:0}: Error finding container 62090f85c3336e5a22c648ae691fae255ccb461fc41e9437b002ab4aa51888d7: Status 404 returned error can't find the container with id 62090f85c3336e5a22c648ae691fae255ccb461fc41e9437b002ab4aa51888d7 Jul 28 16:19:58.612491 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:58.612468 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27acf1b0_1531_400f_8d62_fd525316311d.slice/crio-b22b4d46f06810f3efa11af8a370518c0152b9801898dc4a4d6b4367d0b4301e WatchSource:0}: Error finding container b22b4d46f06810f3efa11af8a370518c0152b9801898dc4a4d6b4367d0b4301e: Status 404 returned error can't find the container with id b22b4d46f06810f3efa11af8a370518c0152b9801898dc4a4d6b4367d0b4301e Jul 28 16:19:58.613223 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:58.613117 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b582ed_07c0_4cf7_9a9c_60424d33558c.slice/crio-c2db6b5f5202b95629378c4f9e113c2f0cbe82c60f4050cb8e34725ce736a89d WatchSource:0}: Error finding container c2db6b5f5202b95629378c4f9e113c2f0cbe82c60f4050cb8e34725ce736a89d: Status 404 returned error can't find the container with id c2db6b5f5202b95629378c4f9e113c2f0cbe82c60f4050cb8e34725ce736a89d Jul 28 16:19:58.615090 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:19:58.614999 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2db18f9_9353_4501_babc_b9f1e28ab745.slice/crio-e63a47a940c19339c05bba3f58be5cc4fdda5eed04668ad7761d124a4599ad3a WatchSource:0}: Error finding container e63a47a940c19339c05bba3f58be5cc4fdda5eed04668ad7761d124a4599ad3a: Status 404 returned error can't find the container with id e63a47a940c19339c05bba3f58be5cc4fdda5eed04668ad7761d124a4599ad3a Jul 28 16:19:58.695407 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.695380 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:58.695508 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:58.695490 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:19:58.706296 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.706187 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" event={"ID":"10b582ed-07c0-4cf7-9a9c-60424d33558c","Type":"ContainerStarted","Data":"c2db6b5f5202b95629378c4f9e113c2f0cbe82c60f4050cb8e34725ce736a89d"} Jul 28 16:19:58.710004 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.709984 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rhfmx" event={"ID":"27acf1b0-1531-400f-8d62-fd525316311d","Type":"ContainerStarted","Data":"b22b4d46f06810f3efa11af8a370518c0152b9801898dc4a4d6b4367d0b4301e"} Jul 28 16:19:58.712721 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.712682 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" event={"ID":"881f0dd5-06fb-4516-a54e-0f11567fcf93","Type":"ContainerStarted","Data":"62090f85c3336e5a22c648ae691fae255ccb461fc41e9437b002ab4aa51888d7"} Jul 28 16:19:58.714414 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.714394 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" event={"ID":"207f737ba8ca766d2b41e867763749e9","Type":"ContainerStarted","Data":"7bd6255714ebbba568e97033e9f2838f16d6b80fc515b8a53faf15b8dc383fe0"} Jul 28 16:19:58.715756 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.715735 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jx5jf" event={"ID":"90c74a1a-35fb-4b6f-b6b8-ab339710936a","Type":"ContainerStarted","Data":"642ff9486d1ebc5b8d84dfbf4a940f2bdbfea33a74ebaa82341503f4630ebe9a"} Jul 28 16:19:58.716873 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.716852 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"ea5c8eefc2ff40bb24a205a9241948db37a9f0d3fbfd1217ad7369bd1425b88f"} Jul 28 16:19:58.719215 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.719195 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pdvc9" event={"ID":"d2db18f9-9353-4501-babc-b9f1e28ab745","Type":"ContainerStarted","Data":"e63a47a940c19339c05bba3f58be5cc4fdda5eed04668ad7761d124a4599ad3a"} Jul 28 16:19:58.723025 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.723004 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerStarted","Data":"8ed3b09e0b1466c90353ed20f4b12cb6acbc2be348adc49c2501965595348c9b"} Jul 28 16:19:58.725135 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.725117 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tbkgq" event={"ID":"e678a606-ebd7-4932-997b-12a94ef7f712","Type":"ContainerStarted","Data":"579684862c1c8b270182be4df5622f06b2abdda23a5810d39a6a9c0a1bdb360e"} Jul 28 16:19:58.726097 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.726078 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jrbqd" event={"ID":"077326cd-7c10-4e5c-9c73-51637330d2c9","Type":"ContainerStarted","Data":"10e947c4fecde3782f2ee20dd3eb13153ea31113a9d00ed8cff61b43c3ec64fe"} Jul 28 16:19:58.731543 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:58.731496 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-37.ec2.internal" podStartSLOduration=2.731480361 podStartE2EDuration="2.731480361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:19:58.731308487 +0000 UTC m=+3.589848231" watchObservedRunningTime="2026-07-28 16:19:58.731480361 +0000 UTC m=+3.590020107" Jul 28 16:19:59.138501 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.137800 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:19:59.138501 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.138013 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:19:59.138501 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.138032 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:19:59.138501 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.138045 2574 projected.go:196] Error preparing data for projected volume kube-api-access-lgjw5 for pod openshift-network-diagnostics/network-check-target-7hxvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:19:59.138501 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.138100 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5 podName:988926e8-4287-4223-a8b2-8f7d813fedfc nodeName:}" failed. No retries permitted until 2026-07-28 16:20:01.13808228 +0000 UTC m=+5.996622018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgjw5" (UniqueName: "kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5") pod "network-check-target-7hxvb" (UID: "988926e8-4287-4223-a8b2-8f7d813fedfc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:19:59.239260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.238622 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:19:59.239260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.238663 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:59.239260 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.238792 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:19:59.239260 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.238854 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret podName:9ab0cb4d-8319-49fc-8031-7e2b24124055 nodeName:}" failed. No retries permitted until 2026-07-28 16:20:01.238835792 +0000 UTC m=+6.097375516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret") pod "global-pull-secret-syncer-4wfgl" (UID: "9ab0cb4d-8319-49fc-8031-7e2b24124055") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:19:59.239260 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.238918 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:19:59.239260 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.238952 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs podName:7600bbc2-65b3-4c9f-87aa-053bf079816c nodeName:}" failed. No retries permitted until 2026-07-28 16:20:01.238940433 +0000 UTC m=+6.097480167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs") pod "network-metrics-daemon-mgt4v" (UID: "7600bbc2-65b3-4c9f-87aa-053bf079816c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:19:59.565384 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.565321 2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 16:14:57 +0000 UTC" deadline="2028-01-24 07:29:33.577796463 +0000 UTC" Jul 28 16:19:59.565384 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.565383 2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13071h9m34.012417715s" Jul 28 16:19:59.697895 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.697864 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:19:59.698050 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.697995 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:19:59.698393 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.698363 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:19:59.698486 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:19:59.698471 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:19:59.734316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.733693 2574 generic.go:356] "Generic (PLEG): container finished" podID="eadeb8c45c8e8f79dc05f80b5906def8" containerID="30205461bef6a928a391a9726c38de4e8f739e53b7a99d3a338bbbc65a1bc16d" exitCode=0 Jul 28 16:19:59.734480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:19:59.733833 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" event={"ID":"eadeb8c45c8e8f79dc05f80b5906def8","Type":"ContainerDied","Data":"30205461bef6a928a391a9726c38de4e8f739e53b7a99d3a338bbbc65a1bc16d"} Jul 28 16:20:00.695763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:00.695730 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:00.696231 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:00.695896 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:00.755934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:00.755872 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" event={"ID":"eadeb8c45c8e8f79dc05f80b5906def8","Type":"ContainerStarted","Data":"3a3115bd1f68cc92fbd7cd7d6e8b660c531f44cc028f4f3abcd75fe1a2e4a65f"} Jul 28 16:20:01.156783 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:01.156679 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:01.156953 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.156862 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:20:01.156953 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.156883 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:20:01.156953 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.156895 2574 projected.go:196] Error preparing data for projected volume kube-api-access-lgjw5 for pod openshift-network-diagnostics/network-check-target-7hxvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:01.157103 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.156951 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5 podName:988926e8-4287-4223-a8b2-8f7d813fedfc nodeName:}" failed. No retries permitted until 2026-07-28 16:20:05.156933121 +0000 UTC m=+10.015472843 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgjw5" (UniqueName: "kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5") pod "network-check-target-7hxvb" (UID: "988926e8-4287-4223-a8b2-8f7d813fedfc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:01.264354 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:01.262899 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:01.264354 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:01.262950 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:01.264354 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.263169 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:01.264354 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.263248 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret podName:9ab0cb4d-8319-49fc-8031-7e2b24124055 nodeName:}" failed. No retries permitted until 2026-07-28 16:20:05.263229721 +0000 UTC m=+10.121769464 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret") pod "global-pull-secret-syncer-4wfgl" (UID: "9ab0cb4d-8319-49fc-8031-7e2b24124055") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:01.264354 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.263772 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:01.264354 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.263831 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs podName:7600bbc2-65b3-4c9f-87aa-053bf079816c nodeName:}" failed. No retries permitted until 2026-07-28 16:20:05.263814981 +0000 UTC m=+10.122354731 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs") pod "network-metrics-daemon-mgt4v" (UID: "7600bbc2-65b3-4c9f-87aa-053bf079816c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:01.697898 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:01.697867 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:01.698272 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.698104 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:01.698538 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:01.698519 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:01.698643 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:01.698622 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:02.695573 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:02.695539 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:02.695785 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:02.695682 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:03.695344 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:03.695315 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:03.695789 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:03.695451 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:03.695789 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:03.695776 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:03.695907 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:03.695882 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:04.695606 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:04.695442 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:04.695606 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:04.695595 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:05.195802 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:05.195771 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:05.196006 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.195985 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:20:05.196083 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.196015 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:20:05.196083 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.196027 2574 projected.go:196] Error preparing data for projected volume kube-api-access-lgjw5 for pod openshift-network-diagnostics/network-check-target-7hxvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:05.196192 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.196086 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5 podName:988926e8-4287-4223-a8b2-8f7d813fedfc nodeName:}" failed. No retries permitted until 2026-07-28 16:20:13.196067893 +0000 UTC m=+18.054607630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgjw5" (UniqueName: "kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5") pod "network-check-target-7hxvb" (UID: "988926e8-4287-4223-a8b2-8f7d813fedfc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:05.296841 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:05.296810 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:05.296841 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:05.296848 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:05.297064 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.296965 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:05.297064 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.297010 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret podName:9ab0cb4d-8319-49fc-8031-7e2b24124055 nodeName:}" failed. No retries permitted until 2026-07-28 16:20:13.296997562 +0000 UTC m=+18.155537283 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret") pod "global-pull-secret-syncer-4wfgl" (UID: "9ab0cb4d-8319-49fc-8031-7e2b24124055") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:05.297552 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.297468 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:05.297552 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.297521 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs podName:7600bbc2-65b3-4c9f-87aa-053bf079816c nodeName:}" failed. No retries permitted until 2026-07-28 16:20:13.297506059 +0000 UTC m=+18.156045798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs") pod "network-metrics-daemon-mgt4v" (UID: "7600bbc2-65b3-4c9f-87aa-053bf079816c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:05.700606 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:05.700578 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:05.700606 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:05.700593 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:05.701102 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.700698 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:05.701102 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:05.700805 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:06.695368 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:06.694876 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:06.695368 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:06.695013 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:07.695132 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:07.695103 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:07.695570 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:07.695219 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:07.695631 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:07.695570 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:07.695683 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:07.695655 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:08.694983 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:08.694954 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:08.695151 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:08.695094 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:09.698932 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:09.698905 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:09.699369 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:09.699007 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:09.699369 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:09.699281 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:09.699369 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:09.699354 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:10.695808 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:10.695773 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:10.695967 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:10.695903 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:11.698395 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:11.698365 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:11.698832 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:11.698366 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:11.698832 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:11.698480 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:11.698832 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:11.698528 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:12.695548 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:12.695520 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:12.695770 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:12.695632 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:13.255143 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:13.255100 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:13.255731 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.255263 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:20:13.255731 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.255285 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:20:13.255731 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.255295 2574 projected.go:196] Error preparing data for projected volume kube-api-access-lgjw5 for pod openshift-network-diagnostics/network-check-target-7hxvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:13.255731 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.255347 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5 podName:988926e8-4287-4223-a8b2-8f7d813fedfc nodeName:}" failed. No retries permitted until 2026-07-28 16:20:29.255333486 +0000 UTC m=+34.113873209 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgjw5" (UniqueName: "kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5") pod "network-check-target-7hxvb" (UID: "988926e8-4287-4223-a8b2-8f7d813fedfc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:13.355829 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:13.355793 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:13.356007 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:13.355839 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:13.356007 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.355942 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:13.356007 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.355953 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:13.356007 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.356007 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret podName:9ab0cb4d-8319-49fc-8031-7e2b24124055 nodeName:}" failed. No retries permitted until 2026-07-28 16:20:29.355987505 +0000 UTC m=+34.214527245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret") pod "global-pull-secret-syncer-4wfgl" (UID: "9ab0cb4d-8319-49fc-8031-7e2b24124055") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:13.356173 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.356026 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs podName:7600bbc2-65b3-4c9f-87aa-053bf079816c nodeName:}" failed. No retries permitted until 2026-07-28 16:20:29.356017074 +0000 UTC m=+34.214556802 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs") pod "network-metrics-daemon-mgt4v" (UID: "7600bbc2-65b3-4c9f-87aa-053bf079816c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:13.695460 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:13.695379 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:13.695623 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:13.695379 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:13.695623 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.695506 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:13.695623 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:13.695574 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:14.695239 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:14.695208 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:14.695762 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:14.695339 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:15.698498 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:15.698468 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:15.698956 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:15.698470 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:15.698956 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:15.698581 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:15.698956 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:15.698648 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:16.695507 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:16.695475 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:16.695693 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:16.695615 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:17.698629 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.698226 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:17.698629 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:17.698371 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:17.698629 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.698454 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:17.698629 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:17.698528 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:17.793188 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.793160 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-tbkgq" event={"ID":"e678a606-ebd7-4932-997b-12a94ef7f712","Type":"ContainerStarted","Data":"e144b7b348cc556d67e3ca0408c080f21afbcc5f67873587561d2856328564a9"} Jul 28 16:20:17.794875 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.794843 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" event={"ID":"10b582ed-07c0-4cf7-9a9c-60424d33558c","Type":"ContainerStarted","Data":"1015cc4aa55ea5012bfde213cfb49032d674a45a3477e3bafabe9383c70bc88f"} Jul 28 16:20:17.796425 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.796385 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rhfmx" event={"ID":"27acf1b0-1531-400f-8d62-fd525316311d","Type":"ContainerStarted","Data":"702c25c65a9bb9857b7f9e73e8d600bf61b60e23c8678938022a4e60c1a59885"} Jul 28 16:20:17.798021 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.797999 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" event={"ID":"881f0dd5-06fb-4516-a54e-0f11567fcf93","Type":"ContainerStarted","Data":"e99e7f94f5e1f11da25265741c2073eb13710be92dabd333c649a1bc8f2993f1"} Jul 28 16:20:17.800055 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.799779 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jx5jf" event={"ID":"90c74a1a-35fb-4b6f-b6b8-ab339710936a","Type":"ContainerStarted","Data":"6a4a7dea256104d0b1f7c9d664c3e31296951e0059c491b6487484633c845127"} Jul 28 16:20:17.802224 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.802026 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"2bdace7fa53e3467f963d56b13e330dbda22caa50b2b6c7e40ba012cb1637e4e"} Jul 28 16:20:17.802224 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.802116 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"3157dbe65fcd73182ba7b4c7d1e03df61ad03b02de5964e7af1134ec34da0122"} Jul 28 16:20:17.803594 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.803568 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pdvc9" event={"ID":"d2db18f9-9353-4501-babc-b9f1e28ab745","Type":"ContainerStarted","Data":"1bb45b21597386f986d3229b614fa3a91495a5b0803d1e13d3cdbcc68a5cac4a"} Jul 28 16:20:17.804748 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.804726 2574 generic.go:356] "Generic (PLEG): container finished" podID="40b096ac-3f93-4ef8-a116-b3599aa57561" containerID="015724adb3b694be2d866d7471dc3625c784badad66f189688ecfbd0f21b6e1e" exitCode=0 Jul 28 16:20:17.804748 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.804737 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerDied","Data":"015724adb3b694be2d866d7471dc3625c784badad66f189688ecfbd0f21b6e1e"} Jul 28 16:20:17.813794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.813751 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-37.ec2.internal" podStartSLOduration=21.813689082 podStartE2EDuration="21.813689082s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:00.775210713 +0000 UTC m=+5.633750457" watchObservedRunningTime="2026-07-28 16:20:17.813689082 +0000 UTC m=+22.672228815" Jul 28 16:20:17.814202 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.814179 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-tbkgq" podStartSLOduration=4.008852349 podStartE2EDuration="22.814169208s" totalImagesPullingTime="18.805316859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:17.81387436 +0000 UTC m=+22.672414099" watchObservedRunningTime="2026-07-28 16:20:17.814169208 +0000 UTC m=+22.672708950" Jul 28 16:20:17.875660 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.875618 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rhfmx" podStartSLOduration=4.081902976 podStartE2EDuration="22.875604755s" totalImagesPullingTime="18.793701779s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:17.855333866 +0000 UTC m=+22.713873609" watchObservedRunningTime="2026-07-28 16:20:17.875604755 +0000 UTC m=+22.734144498" Jul 28 16:20:17.875992 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.875968 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nk8pj" podStartSLOduration=4.054752945 podStartE2EDuration="22.875959859s" totalImagesPullingTime="18.821206914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:17.875350603 +0000 UTC m=+22.733890347" watchObservedRunningTime="2026-07-28 16:20:17.875959859 +0000 UTC m=+22.734499602" Jul 28 16:20:17.892931 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.892898 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jx5jf" podStartSLOduration=12.317248793 podStartE2EDuration="22.89288473s" totalImagesPullingTime="10.575635937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:17.892658231 +0000 UTC m=+22.751197975" watchObservedRunningTime="2026-07-28 16:20:17.89288473 +0000 UTC m=+22.751424472" Jul 28 16:20:17.909183 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:17.909147 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pdvc9" podStartSLOduration=4.11873166 podStartE2EDuration="22.909138674s" totalImagesPullingTime="18.790407014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:17.90910079 +0000 UTC m=+22.767640535" watchObservedRunningTime="2026-07-28 16:20:17.909138674 +0000 UTC m=+22.767678416" Jul 28 16:20:18.016703 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.016673 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:18.089918 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.089896 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:18.398594 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.398512 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:18.495986 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.495951 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:18.557020 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.556993 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:18.695168 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.695097 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:18.695315 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:18.695243 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:18.809997 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.809964 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"97e783ea0f09022ac06873f67bfecba50023ddd4f0941eb310bdc5491127b649"} Jul 28 16:20:18.810804 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.810059 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"e0cd6408420f83f4b73239a14bdc04732bf6bebbf5732b2452f4777aed9fbc6c"} Jul 28 16:20:18.810804 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.810114 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"7535a2e5bf86cea6ce283e0698a9e5c351c06d3a6c6a7a572844e38f132d02c2"} Jul 28 16:20:18.810804 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:18.810161 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"21281834e52a10dd0b9f82db17aac88effa5832663308915055c73efa2c6876e"} Jul 28 16:20:19.199912 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.199883 2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 16:20:19.259780 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.259744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:19.360449 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.360376 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:19.574161 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.574056 2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T16:20:19.199909072Z","UUID":"01c34bda-5149-4897-a681-a323e8bd9f2b","Handler":null,"Name":"","Endpoint":""} Jul 28 16:20:19.575657 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.575635 2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 16:20:19.575807 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.575675 2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 16:20:19.653046 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.652956 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:19.698438 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.698399 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:19.698620 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.698399 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:19.698620 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:19.698533 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:19.698756 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:19.698612 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:19.706157 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.706130 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:19.815078 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.815043 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" event={"ID":"10b582ed-07c0-4cf7-9a9c-60424d33558c","Type":"ContainerStarted","Data":"1babe1a165861b78eaca261a5097e65017b7ed5a229b16ac686a184a9ead1c06"} Jul 28 16:20:19.817568 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.817541 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jrbqd" event={"ID":"077326cd-7c10-4e5c-9c73-51637330d2c9","Type":"ContainerStarted","Data":"9dc641b1c015bbfbdd7046ff0ccdf54f265dff12162c5cb68549b4a9abbbd3e2"} Jul 28 16:20:19.835152 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.835099 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jrbqd" podStartSLOduration=6.052399847 podStartE2EDuration="24.835080267s" totalImagesPullingTime="18.78268042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:19.834080793 +0000 UTC m=+24.692620534" watchObservedRunningTime="2026-07-28 16:20:19.835080267 +0000 UTC m=+24.693620013" Jul 28 16:20:19.937011 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.936936 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:20:19.937582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:19.937563 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:20:20.465335 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:20.465247 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:20.695218 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:20.695188 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:20.695364 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:20.695314 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:20.792990 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:20.792919 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:20.823492 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:20.823457 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"8e9e13f880905c1682dc41ee3ef85938a87312bc9d776d9cbcd6ad56db1de899"} Jul 28 16:20:20.823927 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:20.823737 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:20:20.824227 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:20.824208 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jx5jf" Jul 28 16:20:20.894164 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:20.894134 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:21.098910 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:21.098832 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:21.695492 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:21.695457 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:21.695677 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:21.695589 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:21.695677 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:21.695641 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:21.695816 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:21.695733 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:22.695096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.694925 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:22.695701 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:22.695176 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:22.829286 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.829222 2574 generic.go:356] "Generic (PLEG): container finished" podID="40b096ac-3f93-4ef8-a116-b3599aa57561" containerID="108ab9b30d2c493e2bff5c2fcaceaae71f3f21f10fa80107a4b1d1b1cc795841" exitCode=0 Jul 28 16:20:22.829424 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.829287 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerDied","Data":"108ab9b30d2c493e2bff5c2fcaceaae71f3f21f10fa80107a4b1d1b1cc795841"} Jul 28 16:20:22.831538 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.831513 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" event={"ID":"10b582ed-07c0-4cf7-9a9c-60424d33558c","Type":"ContainerStarted","Data":"aaf1ed4f0646c8792d8572f47666da1c3cca70c5d7f0ab6a2e7d9e84737ad1d1"} Jul 28 16:20:22.835024 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.835003 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" event={"ID":"0216df9b-1601-4dfc-b823-c0d00e69ee69","Type":"ContainerStarted","Data":"26af8a58d8982caf55ea2cb4154a787ee7aa816852c916bb86fd6f2602e9a9f2"} Jul 28 16:20:22.835257 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.835239 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:20:22.835326 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.835317 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:20:22.835377 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.835369 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:20:22.850163 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.850137 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:20:22.850272 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.850259 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:20:22.918923 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:22.918882 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6jjdc" podStartSLOduration=4.838910863 podStartE2EDuration="27.918869657s" totalImagesPullingTime="23.079958794s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:22.879211643 +0000 UTC m=+27.737751386" watchObservedRunningTime="2026-07-28 16:20:22.918869657 +0000 UTC m=+27.777409377" Jul 28 16:20:23.695532 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:23.695217 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:23.695532 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:23.695257 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:23.695971 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:23.695609 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:23.695971 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:23.695755 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:23.839025 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:23.838985 2574 generic.go:356] "Generic (PLEG): container finished" podID="40b096ac-3f93-4ef8-a116-b3599aa57561" containerID="0d633b0a00eb9b94fafd64c47cac1aab0d749ee5966283447331b3dd4a80a4a7" exitCode=0 Jul 28 16:20:23.839194 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:23.839071 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerDied","Data":"0d633b0a00eb9b94fafd64c47cac1aab0d749ee5966283447331b3dd4a80a4a7"} Jul 28 16:20:23.863667 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:23.863614 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" podStartSLOduration=9.856388593 podStartE2EDuration="28.863336423s" totalImagesPullingTime="19.00694783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:22.91983327 +0000 UTC m=+27.778373013" watchObservedRunningTime="2026-07-28 16:20:23.863336423 +0000 UTC m=+28.721876166" Jul 28 16:20:24.406922 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.406894 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:24.462497 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.462463 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4wfgl"] Jul 28 16:20:24.462626 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.462608 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:24.462755 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:24.462728 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:24.468382 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.468364 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7hxvb"] Jul 28 16:20:24.468488 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.468469 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:24.468598 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:24.468580 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:24.469089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.469027 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mgt4v"] Jul 28 16:20:24.469174 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.469152 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:24.469261 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:24.469232 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:24.502868 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.502843 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:24.576162 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:24.576138 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:25.399188 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:25.399160 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:25.455897 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:25.455874 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:25.696269 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:25.696204 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:25.696395 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:25.696284 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:25.790043 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:25.790015 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:25.847846 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:25.847818 2574 generic.go:356] "Generic (PLEG): container finished" podID="40b096ac-3f93-4ef8-a116-b3599aa57561" containerID="9fe5a5b7b8ca8e12b4cffd45c040b9369f24b4b7a916d60dd8ca7ef1b71c21eb" exitCode=0 Jul 28 16:20:25.847957 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:25.847853 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerDied","Data":"9fe5a5b7b8ca8e12b4cffd45c040b9369f24b4b7a916d60dd8ca7ef1b71c21eb"} Jul 28 16:20:25.861097 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:25.861078 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:26.105765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.105687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:26.210172 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.210145 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:26.301974 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.301943 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:26.369901 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.369836 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:26.471564 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.471536 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:26.695827 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.695755 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:26.695827 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.695764 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:26.696025 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:26.695843 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:26.696025 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:26.695989 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:26.953785 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:26.953678 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rhfmx_27acf1b0-1531-400f-8d62-fd525316311d/dns-node-resolver/0.log" Jul 28 16:20:27.695634 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:27.695421 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:27.696106 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:27.695761 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:27.740513 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:27.740475 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:27.841605 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:27.841568 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:27.952444 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:27.952372 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pdvc9_d2db18f9-9353-4501-babc-b9f1e28ab745/node-ca/0.log" Jul 28 16:20:28.642076 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:28.642045 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:28.695438 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:28.695412 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:28.695589 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:28.695542 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:28.695589 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:28.695578 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:28.695683 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:28.695659 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:29.277190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:29.277122 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:29.277345 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.277319 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 16:20:29.277345 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.277343 2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 16:20:29.277428 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.277353 2574 projected.go:196] Error preparing data for projected volume kube-api-access-lgjw5 for pod openshift-network-diagnostics/network-check-target-7hxvb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:29.277428 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.277407 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5 podName:988926e8-4287-4223-a8b2-8f7d813fedfc nodeName:}" failed. No retries permitted until 2026-07-28 16:21:01.2773915 +0000 UTC m=+66.135931238 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lgjw5" (UniqueName: "kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5") pod "network-check-target-7hxvb" (UID: "988926e8-4287-4223-a8b2-8f7d813fedfc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 16:20:29.377916 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:29.377882 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:29.378085 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:29.377923 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:29.378085 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.378042 2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:29.378085 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.378047 2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:29.378206 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.378096 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret podName:9ab0cb4d-8319-49fc-8031-7e2b24124055 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:01.37808268 +0000 UTC m=+66.236622421 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret") pod "global-pull-secret-syncer-4wfgl" (UID: "9ab0cb4d-8319-49fc-8031-7e2b24124055") : object "kube-system"/"original-pull-secret" not registered Jul 28 16:20:29.378206 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.378115 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs podName:7600bbc2-65b3-4c9f-87aa-053bf079816c nodeName:}" failed. No retries permitted until 2026-07-28 16:21:01.378106331 +0000 UTC m=+66.236646052 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs") pod "network-metrics-daemon-mgt4v" (UID: "7600bbc2-65b3-4c9f-87aa-053bf079816c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 16:20:29.695651 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:29.695584 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:29.695797 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:29.695740 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:30.148433 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.148387 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:30.300968 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.300935 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:30.492147 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.489980 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:30.591116 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.591086 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:30.690584 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.690554 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:30.695483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.695463 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:30.695580 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:30.695558 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:30.695618 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.695605 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:30.695700 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:30.695684 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:30.862155 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.862086 2574 generic.go:356] "Generic (PLEG): container finished" podID="40b096ac-3f93-4ef8-a116-b3599aa57561" containerID="e36a8e2e3807b91e9cf1f248285492854fa7e414f10bc17c93cedb4123ee6a57" exitCode=0 Jul 28 16:20:30.862155 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:30.862139 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerDied","Data":"e36a8e2e3807b91e9cf1f248285492854fa7e414f10bc17c93cedb4123ee6a57"} Jul 28 16:20:31.054484 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:31.054457 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:31.257055 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:31.257028 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:31.451259 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:31.451228 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:31.694923 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:31.694843 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:31.695070 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:31.694962 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:31.723727 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:31.723675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:31.866854 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:31.866820 2574 generic.go:356] "Generic (PLEG): container finished" podID="40b096ac-3f93-4ef8-a116-b3599aa57561" containerID="11d4e72e614c8a05f1560365058128264d466e07854c9714fee94aba587b1308" exitCode=0 Jul 28 16:20:31.866854 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:31.866857 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerDied","Data":"11d4e72e614c8a05f1560365058128264d466e07854c9714fee94aba587b1308"} Jul 28 16:20:32.248828 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:32.248729 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:32.688154 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:32.688077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:32.694931 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:32.694905 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:32.694931 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:32.694925 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:32.695036 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:32.695007 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:32.695138 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:32.695121 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:32.850694 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:32.850664 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:32.873055 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:32.873026 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" event={"ID":"40b096ac-3f93-4ef8-a116-b3599aa57561","Type":"ContainerStarted","Data":"130699eb612388b3aa3be05ef805f2fddf056ae57d100afd2a2e31f091f57fc8"} Jul 28 16:20:32.899879 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:32.899834 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8hjhz" podStartSLOduration=8.154757423 podStartE2EDuration="37.899821833s" totalImagesPullingTime="29.74506441s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:32.898642525 +0000 UTC m=+37.757182269" watchObservedRunningTime="2026-07-28 16:20:32.899821833 +0000 UTC m=+37.758361575" Jul 28 16:20:33.043650 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:33.043619 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:33.250275 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:33.250245 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:33.407348 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:33.407261 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:33.485816 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:33.485707 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:33.695188 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:33.695118 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:33.695319 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:33.695218 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:33.772730 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:33.772687 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:33.878182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:33.878156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:34.249837 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:34.249804 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:34.520580 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:34.520544 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:34.631443 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:34.631415 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:34.695278 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:34.695253 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:34.695278 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:34.695284 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:34.695441 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:34.695348 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:34.695534 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:34.695464 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:34.890821 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:34.890743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:35.042493 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:35.042460 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:35.656077 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:35.656043 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:35.696343 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:35.696317 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:35.696816 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:35.696784 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:35.754564 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:35.754529 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:35.855555 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:35.855518 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:36.056633 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:36.056600 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:36.471929 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:36.471899 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:36.535807 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:36.535777 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:36.695102 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:36.695074 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:36.695244 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:36.695074 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:36.695244 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:36.695170 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:36.695313 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:36.695240 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:36.844952 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:36.844882 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:37.262214 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:37.262184 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:37.659010 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:37.658930 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:37.695279 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:37.695259 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:37.695381 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:37.695360 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:37.860590 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:37.860560 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:38.243358 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:38.243318 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:38.695828 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:38.695748 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:38.696181 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:38.695748 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:38.696181 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:38.695848 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:38.696181 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:38.695916 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:39.221289 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:39.221253 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:39.285779 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:39.285745 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:39.659469 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:39.659384 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:39.695217 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:39.695191 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:39.695356 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:39.695320 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:39.906731 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:39.906685 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:40.239103 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:40.239070 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:40.300434 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:40.300399 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:40.695452 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:40.695387 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:40.695560 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:40.695489 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:40.695560 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:40.695547 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:40.695675 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:40.695651 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:40.713149 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:40.713120 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:40.859576 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:40.859550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:41.695375 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:41.695343 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:41.695847 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:41.695443 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:42.694913 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:42.694876 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:42.695079 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:42.694876 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:42.695079 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:42.694981 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:42.695079 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:42.695054 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:43.695800 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:43.695770 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:43.696195 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:43.695876 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:44.695561 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:44.695534 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:44.695724 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:44.695622 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:44.695724 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:44.695677 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:44.695802 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:44.695785 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:45.695570 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:45.695543 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:45.695947 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:45.695625 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:46.694974 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:46.694946 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:46.695137 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:46.694951 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:46.695137 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:46.695032 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7hxvb" podUID="988926e8-4287-4223-a8b2-8f7d813fedfc" Jul 28 16:20:46.695137 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:46.695116 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4wfgl" podUID="9ab0cb4d-8319-49fc-8031-7e2b24124055" Jul 28 16:20:47.003000 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:47.002972 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:47.159991 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:47.159962 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:47.257701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:47.257631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:47.358408 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:47.358313 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:47.456791 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:47.456761 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:47.695175 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:47.695096 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:47.695318 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:20:47.695205 2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mgt4v" podUID="7600bbc2-65b3-4c9f-87aa-053bf079816c" Jul 28 16:20:48.413491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.413460 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:48.475137 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.475116 2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-37.ec2.internal" event="NodeReady" Jul 28 16:20:48.475260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.475218 2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 16:20:48.519278 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.519232 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:48.520287 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.520265 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-668787855d-fgprt"] Jul 28 16:20:48.522168 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.522150 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.524889 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.524865 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mrwcw\"" Jul 28 16:20:48.525002 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.524916 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 16:20:48.525228 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.525210 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 16:20:48.526418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.526386 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 16:20:48.531129 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.531105 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 16:20:48.539311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.539288 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-668787855d-fgprt"] Jul 28 16:20:48.555579 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.555558 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5s6ln"] Jul 28 16:20:48.557571 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.557555 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-95bv8"] Jul 28 16:20:48.557690 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.557673 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.559340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.559323 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.559946 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.559923 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qctb2\"" Jul 28 16:20:48.560077 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.560059 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 16:20:48.560367 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.560348 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 16:20:48.560456 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.560441 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 16:20:48.561646 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.561631 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s22cc\"" Jul 28 16:20:48.580664 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.580640 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 16:20:48.580664 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.580641 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 16:20:48.581634 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.581619 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 16:20:48.581794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.581777 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 16:20:48.611092 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.611036 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-95bv8"] Jul 28 16:20:48.611574 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.611554 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5s6ln"] Jul 28 16:20:48.619299 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619281 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bcb2210-dbb0-4034-9781-77ba34f31757-trusted-ca\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619356 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619308 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bcb2210-dbb0-4034-9781-77ba34f31757-installation-pull-secrets\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619356 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619325 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67ff8958-7dae-4b09-a56e-f001ee724ba2-cert\") pod \"ingress-canary-5s6ln\" (UID: \"67ff8958-7dae-4b09-a56e-f001ee724ba2\") " pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.619423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619354 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bcb2210-dbb0-4034-9781-77ba34f31757-ca-trust-extracted\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619456 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619422 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq8rh\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-kube-api-access-vq8rh\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619489 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619466 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bcb2210-dbb0-4034-9781-77ba34f31757-image-registry-private-configuration\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619489 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619485 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-crio-socket\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.619549 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619507 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk2g8\" (UniqueName: \"kubernetes.io/projected/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-kube-api-access-pk2g8\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.619549 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619530 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptr8m\" (UniqueName: \"kubernetes.io/projected/67ff8958-7dae-4b09-a56e-f001ee724ba2-kube-api-access-ptr8m\") pod \"ingress-canary-5s6ln\" (UID: \"67ff8958-7dae-4b09-a56e-f001ee724ba2\") " pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.619610 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619562 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bcb2210-dbb0-4034-9781-77ba34f31757-registry-certificates\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619610 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619579 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-data-volume\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.619610 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619603 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-registry-tls\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619622 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-bound-sa-token\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.619701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619670 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.619701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.619685 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.669366 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.669345 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r2472"] Jul 28 16:20:48.671754 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.671738 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.675632 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.675617 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 16:20:48.677244 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.677223 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 16:20:48.677526 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.677503 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v7jht\"" Jul 28 16:20:48.694960 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.694940 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:20:48.695163 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.695146 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:20:48.701511 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.701488 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 16:20:48.701777 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.701756 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p4vdz\"" Jul 28 16:20:48.701875 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.701788 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 16:20:48.702658 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.702476 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 16:20:48.702658 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.702559 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r2472"] Jul 28 16:20:48.720210 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720190 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bcb2210-dbb0-4034-9781-77ba34f31757-registry-certificates\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720290 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720213 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-data-volume\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.720290 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720238 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-registry-tls\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720290 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-bound-sa-token\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720290 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720276 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/984ba9f4-1fde-4149-a801-56be5ee472db-config-volume\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720313 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720338 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720362 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/984ba9f4-1fde-4149-a801-56be5ee472db-metrics-tls\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720398 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bcb2210-dbb0-4034-9781-77ba34f31757-trusted-ca\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720421 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bcb2210-dbb0-4034-9781-77ba34f31757-installation-pull-secrets\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720444 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67ff8958-7dae-4b09-a56e-f001ee724ba2-cert\") pod \"ingress-canary-5s6ln\" (UID: \"67ff8958-7dae-4b09-a56e-f001ee724ba2\") " pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720469 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bcb2210-dbb0-4034-9781-77ba34f31757-ca-trust-extracted\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720492 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq8rh\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-kube-api-access-vq8rh\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720517 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bcb2210-dbb0-4034-9781-77ba34f31757-image-registry-private-configuration\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720534 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-crio-socket\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720548 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk2g8\" (UniqueName: \"kubernetes.io/projected/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-kube-api-access-pk2g8\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720548 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-data-volume\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720563 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/984ba9f4-1fde-4149-a801-56be5ee472db-tmp-dir\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720598 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hdjp\" (UniqueName: \"kubernetes.io/projected/984ba9f4-1fde-4149-a801-56be5ee472db-kube-api-access-6hdjp\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720631 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptr8m\" (UniqueName: \"kubernetes.io/projected/67ff8958-7dae-4b09-a56e-f001ee724ba2-kube-api-access-ptr8m\") pod \"ingress-canary-5s6ln\" (UID: \"67ff8958-7dae-4b09-a56e-f001ee724ba2\") " pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.720881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720855 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.721231 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.720944 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-crio-socket\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.721231 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.721138 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bcb2210-dbb0-4034-9781-77ba34f31757-registry-certificates\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.721231 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.721186 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bcb2210-dbb0-4034-9781-77ba34f31757-ca-trust-extracted\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.721956 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.721934 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bcb2210-dbb0-4034-9781-77ba34f31757-trusted-ca\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.724495 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.724474 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bcb2210-dbb0-4034-9781-77ba34f31757-installation-pull-secrets\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.724656 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.724637 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67ff8958-7dae-4b09-a56e-f001ee724ba2-cert\") pod \"ingress-canary-5s6ln\" (UID: \"67ff8958-7dae-4b09-a56e-f001ee724ba2\") " pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.724702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.724637 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-registry-tls\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.724702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.724642 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bcb2210-dbb0-4034-9781-77ba34f31757-image-registry-private-configuration\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.724951 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.724932 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.733491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.733456 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-bound-sa-token\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.733905 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.733885 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq8rh\" (UniqueName: \"kubernetes.io/projected/8bcb2210-dbb0-4034-9781-77ba34f31757-kube-api-access-vq8rh\") pod \"image-registry-668787855d-fgprt\" (UID: \"8bcb2210-dbb0-4034-9781-77ba34f31757\") " pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.734239 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.734219 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptr8m\" (UniqueName: \"kubernetes.io/projected/67ff8958-7dae-4b09-a56e-f001ee724ba2-kube-api-access-ptr8m\") pod \"ingress-canary-5s6ln\" (UID: \"67ff8958-7dae-4b09-a56e-f001ee724ba2\") " pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.734916 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.734898 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk2g8\" (UniqueName: \"kubernetes.io/projected/6ba3bcb8-00f0-478d-bde4-60cbb9c5324d-kube-api-access-pk2g8\") pod \"insights-runtime-extractor-95bv8\" (UID: \"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d\") " pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.820837 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.820818 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/984ba9f4-1fde-4149-a801-56be5ee472db-config-volume\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.820911 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.820854 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/984ba9f4-1fde-4149-a801-56be5ee472db-metrics-tls\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.820911 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.820890 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/984ba9f4-1fde-4149-a801-56be5ee472db-tmp-dir\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.821061 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.820912 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hdjp\" (UniqueName: \"kubernetes.io/projected/984ba9f4-1fde-4149-a801-56be5ee472db-kube-api-access-6hdjp\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.821209 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.821189 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/984ba9f4-1fde-4149-a801-56be5ee472db-tmp-dir\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.821316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.821300 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/984ba9f4-1fde-4149-a801-56be5ee472db-config-volume\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.822806 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.822790 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/984ba9f4-1fde-4149-a801-56be5ee472db-metrics-tls\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.828849 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.828824 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:48.829158 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.829140 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hdjp\" (UniqueName: \"kubernetes.io/projected/984ba9f4-1fde-4149-a801-56be5ee472db-kube-api-access-6hdjp\") pod \"dns-default-r2472\" (UID: \"984ba9f4-1fde-4149-a801-56be5ee472db\") " pod="openshift-dns/dns-default-r2472" Jul 28 16:20:48.833063 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.833040 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:48.867576 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.867529 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5s6ln" Jul 28 16:20:48.871811 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.871789 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-95bv8" Jul 28 16:20:48.931141 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.931047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:48.979963 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:48.979940 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r2472" Jul 28 16:20:49.000186 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.000116 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-668787855d-fgprt"] Jul 28 16:20:49.006593 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:20:49.006561 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bcb2210_dbb0_4034_9781_77ba34f31757.slice/crio-9323376617e45ec165cb215fac4a6bb60670105405d5580710ca45c2e49001f8 WatchSource:0}: Error finding container 9323376617e45ec165cb215fac4a6bb60670105405d5580710ca45c2e49001f8: Status 404 returned error can't find the container with id 9323376617e45ec165cb215fac4a6bb60670105405d5580710ca45c2e49001f8 Jul 28 16:20:49.019671 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.019632 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5s6ln"] Jul 28 16:20:49.022868 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:20:49.022842 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ff8958_7dae_4b09_a56e_f001ee724ba2.slice/crio-49be84b98d6c0c6dbc84f035279b13f173088881a7108b66b5b287590f73987d WatchSource:0}: Error finding container 49be84b98d6c0c6dbc84f035279b13f173088881a7108b66b5b287590f73987d: Status 404 returned error can't find the container with id 49be84b98d6c0c6dbc84f035279b13f173088881a7108b66b5b287590f73987d Jul 28 16:20:49.036443 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.036052 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-95bv8"] Jul 28 16:20:49.043787 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:20:49.043195 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba3bcb8_00f0_478d_bde4_60cbb9c5324d.slice/crio-8e948ddd3fa27d6e718135c2c6c874a16cb1f547874855769e94c64d3ba8f330 WatchSource:0}: Error finding container 8e948ddd3fa27d6e718135c2c6c874a16cb1f547874855769e94c64d3ba8f330: Status 404 returned error can't find the container with id 8e948ddd3fa27d6e718135c2c6c874a16cb1f547874855769e94c64d3ba8f330 Jul 28 16:20:49.113459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.113427 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r2472"] Jul 28 16:20:49.118300 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:20:49.118254 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984ba9f4_1fde_4149_a801_56be5ee472db.slice/crio-edd27bd51397f015f751a500e5df26985d92cabaed276beb38390b2a11562c56 WatchSource:0}: Error finding container edd27bd51397f015f751a500e5df26985d92cabaed276beb38390b2a11562c56: Status 404 returned error can't find the container with id edd27bd51397f015f751a500e5df26985d92cabaed276beb38390b2a11562c56 Jul 28 16:20:49.698794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.698052 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:20:49.703093 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.703071 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-njslf\"" Jul 28 16:20:49.703251 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.703229 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 16:20:49.915995 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.915660 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-668787855d-fgprt" event={"ID":"8bcb2210-dbb0-4034-9781-77ba34f31757","Type":"ContainerStarted","Data":"1a6d07d1f2831d01529e4f918961c30325162cdfab13f6093269aa3fe91308c3"} Jul 28 16:20:49.915995 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.915738 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-668787855d-fgprt" event={"ID":"8bcb2210-dbb0-4034-9781-77ba34f31757","Type":"ContainerStarted","Data":"9323376617e45ec165cb215fac4a6bb60670105405d5580710ca45c2e49001f8"} Jul 28 16:20:49.915995 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.915785 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:20:49.917621 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.917592 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5s6ln" event={"ID":"67ff8958-7dae-4b09-a56e-f001ee724ba2","Type":"ContainerStarted","Data":"49be84b98d6c0c6dbc84f035279b13f173088881a7108b66b5b287590f73987d"} Jul 28 16:20:49.921239 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.921214 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2472" event={"ID":"984ba9f4-1fde-4149-a801-56be5ee472db","Type":"ContainerStarted","Data":"edd27bd51397f015f751a500e5df26985d92cabaed276beb38390b2a11562c56"} Jul 28 16:20:49.924706 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.924648 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-95bv8" event={"ID":"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d","Type":"ContainerStarted","Data":"8e948ddd3fa27d6e718135c2c6c874a16cb1f547874855769e94c64d3ba8f330"} Jul 28 16:20:49.939027 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:49.938466 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-668787855d-fgprt" podStartSLOduration=3.938445935 podStartE2EDuration="3.938445935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:20:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:49.936796549 +0000 UTC m=+54.795336292" watchObservedRunningTime="2026-07-28 16:20:49.938445935 +0000 UTC m=+54.796985680" Jul 28 16:20:52.509058 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:52.509017 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:52.661483 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:52.661448 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:53.112779 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:53.112743 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:53.190101 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:53.190068 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:54.867307 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:54.867280 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bqh6d" Jul 28 16:20:55.037754 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.037705 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:55.134016 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.133986 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:55.943491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.943445 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-95bv8" event={"ID":"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d","Type":"ContainerStarted","Data":"9e2441a9eaaafd2c43766eca12c3253395eedc3a25d5c45b1116dbdbb9755ff2"} Jul 28 16:20:55.945696 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.945427 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5s6ln" event={"ID":"67ff8958-7dae-4b09-a56e-f001ee724ba2","Type":"ContainerStarted","Data":"e01b3a408f261fddbf08b9fec42449bb086c401255d3da1b12b3b5a207aa8a62"} Jul 28 16:20:55.948304 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.948279 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2472" event={"ID":"984ba9f4-1fde-4149-a801-56be5ee472db","Type":"ContainerStarted","Data":"d1fdb7530c11ed5440b8a004865bcbb15839488f2ad65206061aaff1e67cc0b7"} Jul 28 16:20:55.948421 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.948329 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r2472" event={"ID":"984ba9f4-1fde-4149-a801-56be5ee472db","Type":"ContainerStarted","Data":"714506c765d3b13d2a3c9f74cbd1a4038fb3614cfcadb10dcfbb52bdfd7bc0e0"} Jul 28 16:20:55.948528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.948443 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r2472" Jul 28 16:20:55.965433 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.965391 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5s6ln" podStartSLOduration=1.907975606 podStartE2EDuration="7.965377686s" totalImagesPullingTime="6.05740208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:55.96472859 +0000 UTC m=+60.823268334" watchObservedRunningTime="2026-07-28 16:20:55.965377686 +0000 UTC m=+60.823917428" Jul 28 16:20:55.984608 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:55.984561 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-r2472" podStartSLOduration=2.022732932 podStartE2EDuration="7.984544408s" totalImagesPullingTime="5.961811476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:20:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:20:55.983627574 +0000 UTC m=+60.842167319" watchObservedRunningTime="2026-07-28 16:20:55.984544408 +0000 UTC m=+60.843084150" Jul 28 16:20:58.962682 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:58.962606 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-95bv8" event={"ID":"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d","Type":"ContainerStarted","Data":"387d4b2185b79ff62cd4a512e9de7b63e347f77c308c9fbf59567952d65d9b98"} Jul 28 16:20:59.888098 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:59.888018 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:20:59.991013 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:20:59.990979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:00.066883 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.066849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:00.069682 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.069512 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl"] Jul 28 16:21:00.071567 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.071542 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:00.074063 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.074039 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 28 16:21:00.074349 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.074325 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-s8hgz\"" Jul 28 16:21:00.082358 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.082334 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl"] Jul 28 16:21:00.204354 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.204286 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-jt2dl\" (UID: \"42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:00.305042 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.305018 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-jt2dl\" (UID: \"42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:00.305165 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:21:00.305150 2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 28 16:21:00.305230 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:21:00.305218 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0-tls-certificates podName:42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0 nodeName:}" failed. No retries permitted until 2026-07-28 16:21:00.805199695 +0000 UTC m=+65.663739418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" (UID: "42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0") : secret "prometheus-operator-admission-webhook-tls" not found Jul 28 16:21:00.402833 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.402802 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:00.657023 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.656990 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:00.724413 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.724384 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:00.798128 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.798098 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:00.809513 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.809480 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-jt2dl\" (UID: \"42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:00.812398 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.812370 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-jt2dl\" (UID: \"42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:00.901765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.901729 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:00.983203 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:00.983178 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:01.314680 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.314595 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:21:01.317859 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.317827 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 16:21:01.328337 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.328306 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 16:21:01.339380 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.339357 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgjw5\" (UniqueName: \"kubernetes.io/projected/988926e8-4287-4223-a8b2-8f7d813fedfc-kube-api-access-lgjw5\") pod \"network-check-target-7hxvb\" (UID: \"988926e8-4287-4223-a8b2-8f7d813fedfc\") " pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:21:01.415471 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.415442 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:21:01.415611 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.415485 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:21:01.418438 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.418411 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 16:21:01.418558 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.418482 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 16:21:01.428159 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.428131 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ab0cb4d-8319-49fc-8031-7e2b24124055-original-pull-secret\") pod \"global-pull-secret-syncer-4wfgl\" (UID: \"9ab0cb4d-8319-49fc-8031-7e2b24124055\") " pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:21:01.428271 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.428229 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7600bbc2-65b3-4c9f-87aa-053bf079816c-metrics-certs\") pod \"network-metrics-daemon-mgt4v\" (UID: \"7600bbc2-65b3-4c9f-87aa-053bf079816c\") " pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:21:01.607061 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.606990 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p4vdz\"" Jul 28 16:21:01.609129 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.609105 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4wfgl" Jul 28 16:21:01.614869 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.614836 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:21:01.715078 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.715058 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-njslf\"" Jul 28 16:21:01.722311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:01.722288 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mgt4v" Jul 28 16:21:02.381266 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.381237 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7hxvb"] Jul 28 16:21:02.384360 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:02.384328 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988926e8_4287_4223_a8b2_8f7d813fedfc.slice/crio-4331e4ff50a936410e1f5b5424266d0eefc08828878e65f84cd6a75ec4481727 WatchSource:0}: Error finding container 4331e4ff50a936410e1f5b5424266d0eefc08828878e65f84cd6a75ec4481727: Status 404 returned error can't find the container with id 4331e4ff50a936410e1f5b5424266d0eefc08828878e65f84cd6a75ec4481727 Jul 28 16:21:02.602361 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.602289 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mgt4v"] Jul 28 16:21:02.606475 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:02.606427 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7600bbc2_65b3_4c9f_87aa_053bf079816c.slice/crio-b578bf4ec41c62b6ca76a9967ed97cf39dc5bf710b8b0fc7686d49eec2932a9b WatchSource:0}: Error finding container b578bf4ec41c62b6ca76a9967ed97cf39dc5bf710b8b0fc7686d49eec2932a9b: Status 404 returned error can't find the container with id b578bf4ec41c62b6ca76a9967ed97cf39dc5bf710b8b0fc7686d49eec2932a9b Jul 28 16:21:02.606585 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.606511 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl"] Jul 28 16:21:02.611490 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.611459 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4wfgl"] Jul 28 16:21:02.614185 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:02.614085 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c0c206_5a4d_42e4_9d45_b0d6b9e9f2a0.slice/crio-b22896909165fe1fe8fbf26785a6c37c1e92d7e27da3d8a2789617e0798b1f7c WatchSource:0}: Error finding container b22896909165fe1fe8fbf26785a6c37c1e92d7e27da3d8a2789617e0798b1f7c: Status 404 returned error can't find the container with id b22896909165fe1fe8fbf26785a6c37c1e92d7e27da3d8a2789617e0798b1f7c Jul 28 16:21:02.614913 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:02.614894 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab0cb4d_8319_49fc_8031_7e2b24124055.slice/crio-808a3bca29a583e65f60c4345606e1c57125f95099cd32467f548583afb5be18 WatchSource:0}: Error finding container 808a3bca29a583e65f60c4345606e1c57125f95099cd32467f548583afb5be18: Status 404 returned error can't find the container with id 808a3bca29a583e65f60c4345606e1c57125f95099cd32467f548583afb5be18 Jul 28 16:21:02.976690 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.976656 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mgt4v" event={"ID":"7600bbc2-65b3-4c9f-87aa-053bf079816c","Type":"ContainerStarted","Data":"b578bf4ec41c62b6ca76a9967ed97cf39dc5bf710b8b0fc7686d49eec2932a9b"} Jul 28 16:21:02.978064 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.978035 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7hxvb" event={"ID":"988926e8-4287-4223-a8b2-8f7d813fedfc","Type":"ContainerStarted","Data":"4331e4ff50a936410e1f5b5424266d0eefc08828878e65f84cd6a75ec4481727"} Jul 28 16:21:02.979339 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.979312 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" event={"ID":"42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0","Type":"ContainerStarted","Data":"b22896909165fe1fe8fbf26785a6c37c1e92d7e27da3d8a2789617e0798b1f7c"} Jul 28 16:21:02.980530 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.980501 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4wfgl" event={"ID":"9ab0cb4d-8319-49fc-8031-7e2b24124055","Type":"ContainerStarted","Data":"808a3bca29a583e65f60c4345606e1c57125f95099cd32467f548583afb5be18"} Jul 28 16:21:02.982789 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:02.982763 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-95bv8" event={"ID":"6ba3bcb8-00f0-478d-bde4-60cbb9c5324d","Type":"ContainerStarted","Data":"069eb0b9634701af27dfa571cae349527acb966e3af3a9ffd3605ed9b975bfac"} Jul 28 16:21:03.007218 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:03.007170 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-95bv8" podStartSLOduration=1.8100277839999999 podStartE2EDuration="15.007151089s" totalImagesPullingTime="13.197123305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:20:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:03.006238911 +0000 UTC m=+67.864778655" watchObservedRunningTime="2026-07-28 16:21:03.007151089 +0000 UTC m=+67.865690832" Jul 28 16:21:03.402560 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:03.402473 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:03.473549 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:03.473505 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:03.991776 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:03.991383 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mgt4v" event={"ID":"7600bbc2-65b3-4c9f-87aa-053bf079816c","Type":"ContainerStarted","Data":"b152d8cf0dc988d94d3aaf6925ce2c748442e811d2674eb114738d4590f6c503"} Jul 28 16:21:03.991776 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:03.991449 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mgt4v" event={"ID":"7600bbc2-65b3-4c9f-87aa-053bf079816c","Type":"ContainerStarted","Data":"4218e37b41065f1f0cfc5695cdac1eee46ad5b6af9be94ecd67565a1dedb3398"} Jul 28 16:21:04.010948 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:04.010900 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mgt4v" podStartSLOduration=67.895788917 podStartE2EDuration="1m9.010881824s" totalImagesPullingTime="1.115092907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:04.00982109 +0000 UTC m=+68.868360835" watchObservedRunningTime="2026-07-28 16:21:04.010881824 +0000 UTC m=+68.869421568" Jul 28 16:21:04.134654 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:04.134587 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:04.230644 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:04.230597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:04.997626 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:04.997591 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" event={"ID":"42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0","Type":"ContainerStarted","Data":"ef6e9be2aa21d127b43c4080ee27908e14199bbae9d40f537774549c82cbdb0b"} Jul 28 16:21:04.998083 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:04.997978 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:05.003598 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:05.003545 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" Jul 28 16:21:05.016144 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:05.016100 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-jt2dl" podStartSLOduration=3.110222907 podStartE2EDuration="5.016080985s" totalImagesPullingTime="1.905858078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:05.014423055 +0000 UTC m=+69.872962790" watchObservedRunningTime="2026-07-28 16:21:05.016080985 +0000 UTC m=+69.874620729" Jul 28 16:21:05.953824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:05.953788 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r2472" Jul 28 16:21:06.111948 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:06.111913 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:06.195125 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:06.195089 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:06.261340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:06.261307 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:08.011537 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:08.011495 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7hxvb" event={"ID":"988926e8-4287-4223-a8b2-8f7d813fedfc","Type":"ContainerStarted","Data":"6a65255291031bdfc91e9d0570d7b16cb954896113bbcbdeceb78953e2b8f951"} Jul 28 16:21:08.012030 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:08.011611 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:21:08.013072 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:08.013038 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4wfgl" event={"ID":"9ab0cb4d-8319-49fc-8031-7e2b24124055","Type":"ContainerStarted","Data":"68dcf4b79e1a6556cf80fcdf70ad61417d122cecf369270884d192419e3af311"} Jul 28 16:21:08.032544 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:08.032496 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7hxvb" podStartSLOduration=68.382774321 podStartE2EDuration="1m13.032483469s" totalImagesPullingTime="4.649709148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:08.031699028 +0000 UTC m=+72.890238794" watchObservedRunningTime="2026-07-28 16:21:08.032483469 +0000 UTC m=+72.891023211" Jul 28 16:21:08.050238 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:08.050188 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4wfgl" podStartSLOduration=68.626967441 podStartE2EDuration="1m13.050174092s" totalImagesPullingTime="4.423206651s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:08.049681876 +0000 UTC m=+72.908221621" watchObservedRunningTime="2026-07-28 16:21:08.050174092 +0000 UTC m=+72.908713835" Jul 28 16:21:08.837945 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:08.837902 2574 patch_prober.go:28] interesting pod/image-registry-668787855d-fgprt container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 28 16:21:08.838074 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:08.837958 2574 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-668787855d-fgprt" podUID="8bcb2210-dbb0-4034-9781-77ba34f31757" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:21:09.949797 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:09.949744 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:10.894678 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:10.894626 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:10.932477 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:10.932444 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-668787855d-fgprt" Jul 28 16:21:10.977859 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:10.977817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:11.088761 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:11.088729 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:11.186150 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:11.186083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:11.290832 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:11.290801 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:11.391320 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:11.391285 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:11.456970 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:11.456888 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:11.772696 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:11.772675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:11.840010 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:11.839976 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:12.027559 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.027483 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:12.191615 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.191578 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:12.341062 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.341009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:12.451707 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.451685 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:12.713619 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.713592 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l"] Jul 28 16:21:12.716379 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.716360 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.719691 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.719667 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 16:21:12.719821 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.719697 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 16:21:12.720825 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.720806 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 28 16:21:12.720879 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.720861 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 28 16:21:12.720942 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.720929 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 16:21:12.721048 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.721026 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-vtk8f\"" Jul 28 16:21:12.736642 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.736606 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l"] Jul 28 16:21:12.754193 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.754168 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sswf6"] Jul 28 16:21:12.756790 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.756773 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.759770 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.759752 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 16:21:12.760510 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.760487 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 16:21:12.760601 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.760547 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 16:21:12.760741 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.760727 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-drlz2\"" Jul 28 16:21:12.771358 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.771337 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz"] Jul 28 16:21:12.773415 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.773401 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:12.778093 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.778068 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 28 16:21:12.778184 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.778096 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 28 16:21:12.778725 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.778578 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nx4mx\"" Jul 28 16:21:12.783425 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.783408 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 28 16:21:12.796743 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.796702 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz"] Jul 28 16:21:12.804624 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.804604 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9670e41-08a9-4823-88b1-fceaaff248eb-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.804721 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.804647 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9670e41-08a9-4823-88b1-fceaaff248eb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.804721 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.804675 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9670e41-08a9-4823-88b1-fceaaff248eb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.804721 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.804693 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlt6k\" (UniqueName: \"kubernetes.io/projected/f9670e41-08a9-4823-88b1-fceaaff248eb-kube-api-access-mlt6k\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.905950 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.905913 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9670e41-08a9-4823-88b1-fceaaff248eb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.905950 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.905943 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-root\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906143 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.905961 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/db947547-78c7-4d66-b421-46820162100b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:12.906143 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.905980 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlt6k\" (UniqueName: \"kubernetes.io/projected/f9670e41-08a9-4823-88b1-fceaaff248eb-kube-api-access-mlt6k\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.906143 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.905999 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9670e41-08a9-4823-88b1-fceaaff248eb-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.906143 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906063 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906143 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906115 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-tls\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4wvn\" (UniqueName: \"kubernetes.io/projected/db947547-78c7-4d66-b421-46820162100b-kube-api-access-g4wvn\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906180 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9670e41-08a9-4823-88b1-fceaaff248eb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906208 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db947547-78c7-4d66-b421-46820162100b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906237 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-textfile\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906262 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42cjz\" (UniqueName: \"kubernetes.io/projected/2a86c24c-ca67-4414-a724-a27350279c9c-kube-api-access-42cjz\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906287 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906312 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-sys\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906336 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a86c24c-ca67-4414-a724-a27350279c9c-metrics-client-ca\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906388 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906387 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:12.906741 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906446 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:12.906741 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906482 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-wtmp\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906741 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906509 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:12.906741 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.906655 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9670e41-08a9-4823-88b1-fceaaff248eb-metrics-client-ca\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.908196 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.908176 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9670e41-08a9-4823-88b1-fceaaff248eb-openshift-state-metrics-tls\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.908316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.908298 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9670e41-08a9-4823-88b1-fceaaff248eb-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:12.916935 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:12.916915 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlt6k\" (UniqueName: \"kubernetes.io/projected/f9670e41-08a9-4823-88b1-fceaaff248eb-kube-api-access-mlt6k\") pod \"openshift-state-metrics-b8f49f5c6-b852l\" (UID: \"f9670e41-08a9-4823-88b1-fceaaff248eb\") " pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:13.007268 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007218 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-root\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007268 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007246 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/db947547-78c7-4d66-b421-46820162100b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.007418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007268 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007289 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-tls\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007318 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4wvn\" (UniqueName: \"kubernetes.io/projected/db947547-78c7-4d66-b421-46820162100b-kube-api-access-g4wvn\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.007418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007323 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-root\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007367 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db947547-78c7-4d66-b421-46820162100b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.007418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007391 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-textfile\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007409 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42cjz\" (UniqueName: \"kubernetes.io/projected/2a86c24c-ca67-4414-a724-a27350279c9c-kube-api-access-42cjz\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007429 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007459 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-sys\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007485 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a86c24c-ca67-4414-a724-a27350279c9c-metrics-client-ca\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007514 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007583 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-wtmp\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007605 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007609 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-sys\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.007760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007675 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/db947547-78c7-4d66-b421-46820162100b-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.008190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007853 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-wtmp\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.008190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.007962 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.008190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.008111 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db947547-78c7-4d66-b421-46820162100b-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.008190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.008120 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-textfile\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.008370 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.008250 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2a86c24c-ca67-4414-a724-a27350279c9c-metrics-client-ca\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.008725 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.008680 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.009809 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.009783 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-tls\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.009953 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.009935 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.010097 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.010080 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/db947547-78c7-4d66-b421-46820162100b-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.010367 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.010346 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2a86c24c-ca67-4414-a724-a27350279c9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.020228 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.020210 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42cjz\" (UniqueName: \"kubernetes.io/projected/2a86c24c-ca67-4414-a724-a27350279c9c-kube-api-access-42cjz\") pod \"node-exporter-sswf6\" (UID: \"2a86c24c-ca67-4414-a724-a27350279c9c\") " pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.022347 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.022326 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4wvn\" (UniqueName: \"kubernetes.io/projected/db947547-78c7-4d66-b421-46820162100b-kube-api-access-g4wvn\") pod \"kube-state-metrics-56d9f944b4-n5jsz\" (UID: \"db947547-78c7-4d66-b421-46820162100b\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.026155 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.026140 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" Jul 28 16:21:13.065682 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.065655 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sswf6" Jul 28 16:21:13.081516 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.081494 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" Jul 28 16:21:13.175131 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.175097 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l"] Jul 28 16:21:13.178016 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:13.177991 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9670e41_08a9_4823_88b1_fceaaff248eb.slice/crio-5e3f2847e0c64c8e393b382239cda40932b1322d136b14c275800015750f0058 WatchSource:0}: Error finding container 5e3f2847e0c64c8e393b382239cda40932b1322d136b14c275800015750f0058: Status 404 returned error can't find the container with id 5e3f2847e0c64c8e393b382239cda40932b1322d136b14c275800015750f0058 Jul 28 16:21:13.225202 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.225172 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz"] Jul 28 16:21:13.228330 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:13.228307 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb947547_78c7_4d66_b421_46820162100b.slice/crio-91e1ea7380a4c876727e2a1766f5bf3b50586ddf4881fc01f246137d989e7e83 WatchSource:0}: Error finding container 91e1ea7380a4c876727e2a1766f5bf3b50586ddf4881fc01f246137d989e7e83: Status 404 returned error can't find the container with id 91e1ea7380a4c876727e2a1766f5bf3b50586ddf4881fc01f246137d989e7e83 Jul 28 16:21:13.244593 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.244567 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:13.341579 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.341554 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:13.445837 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.445808 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:13.559489 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.559433 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:13.661168 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.661135 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:13.672745 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.672704 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:21:13.675505 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.675484 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.679013 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.678992 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 16:21:13.679139 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.679120 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 16:21:13.679299 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.679212 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 16:21:13.679763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.679571 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 16:21:13.679763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.679573 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vb46g\"" Jul 28 16:21:13.679763 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.679685 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 16:21:13.679941 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.679916 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 16:21:13.680186 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.680167 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 16:21:13.680271 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.680205 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 16:21:13.680516 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.680499 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 16:21:13.699581 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.699563 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:21:13.814277 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814277 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814249 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814476 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814279 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814476 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814356 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814476 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svrdt\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-kube-api-access-svrdt\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814647 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814473 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814647 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814524 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-volume\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814647 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814552 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814647 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814595 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814665 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-out\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814691 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-web-config\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814731 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.814881 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.814838 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.864724 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.864675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916078 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-out\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916118 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-web-config\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916144 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916199 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916227 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916274 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916484 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916523 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svrdt\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-kube-api-access-svrdt\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916565 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916618 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-volume\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916642 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.916702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.916674 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.917486 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.917001 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.917486 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.917088 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.917936 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.917907 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.919784 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.919763 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.924162 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.924135 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.924542 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.924520 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.924624 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.924538 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-web-config\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.925127 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.925068 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-out\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.925127 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.925108 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.925284 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.925167 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-volume\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.925284 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.925178 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.926808 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.926786 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.929816 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.929789 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svrdt\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-kube-api-access-svrdt\") pod \"alertmanager-main-0\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:13.963815 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.963758 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:13.986206 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:13.986181 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:21:14.035828 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:14.035752 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" event={"ID":"db947547-78c7-4d66-b421-46820162100b","Type":"ContainerStarted","Data":"91e1ea7380a4c876727e2a1766f5bf3b50586ddf4881fc01f246137d989e7e83"} Jul 28 16:21:14.039115 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:14.039004 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" event={"ID":"f9670e41-08a9-4823-88b1-fceaaff248eb","Type":"ContainerStarted","Data":"4886de299ee7b4e45359e9845196f15a13c1a04ad26c78e3e00538a86a032f50"} Jul 28 16:21:14.039115 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:14.039072 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" event={"ID":"f9670e41-08a9-4823-88b1-fceaaff248eb","Type":"ContainerStarted","Data":"da93bccd33b3fdcd90d119e2b431fe08afab7855165a373c842ef0171b289738"} Jul 28 16:21:14.039115 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:14.039105 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" event={"ID":"f9670e41-08a9-4823-88b1-fceaaff248eb","Type":"ContainerStarted","Data":"5e3f2847e0c64c8e393b382239cda40932b1322d136b14c275800015750f0058"} Jul 28 16:21:14.040773 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:14.040746 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sswf6" event={"ID":"2a86c24c-ca67-4414-a724-a27350279c9c","Type":"ContainerStarted","Data":"bb32e62fb44a58089ac42b641baa2514c95ed538cd40792ccd537d6e5519cf7d"} Jul 28 16:21:14.172005 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:14.171908 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:21:14.286034 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:14.285999 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54a2ada_8d99_4719_8c69_4d8c6d1c3961.slice/crio-5d0ac26b57e914546de1635a55bf3f2f2cded1a945fd7dbb341a672b146b159a WatchSource:0}: Error finding container 5d0ac26b57e914546de1635a55bf3f2f2cded1a945fd7dbb341a672b146b159a: Status 404 returned error can't find the container with id 5d0ac26b57e914546de1635a55bf3f2f2cded1a945fd7dbb341a672b146b159a Jul 28 16:21:15.045878 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.045795 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" event={"ID":"db947547-78c7-4d66-b421-46820162100b","Type":"ContainerStarted","Data":"065f6d8f407e041d3328922a7606aa347a463646cd99b9efe9fecef8c9ceebbc"} Jul 28 16:21:15.046076 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.045902 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" event={"ID":"db947547-78c7-4d66-b421-46820162100b","Type":"ContainerStarted","Data":"de664cb64673aec353ba3c99bcb744e100d70dfcbf2a5419357311609b93a4b9"} Jul 28 16:21:15.046076 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.045933 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" event={"ID":"db947547-78c7-4d66-b421-46820162100b","Type":"ContainerStarted","Data":"4d2b6067759dc0e5eb3f5599971c10c4b019166500be600177dabec305027b96"} Jul 28 16:21:15.048190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.048160 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" event={"ID":"f9670e41-08a9-4823-88b1-fceaaff248eb","Type":"ContainerStarted","Data":"c1558bd2ba0f59c3127edde4d9ae502323aec413addc7d3423533ac5f1bdb00b"} Jul 28 16:21:15.049556 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.049521 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerStarted","Data":"5d0ac26b57e914546de1635a55bf3f2f2cded1a945fd7dbb341a672b146b159a"} Jul 28 16:21:15.051362 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.051336 2574 generic.go:356] "Generic (PLEG): container finished" podID="2a86c24c-ca67-4414-a724-a27350279c9c" containerID="761e5e2a23ff336fbdb186093af9a813458a6c63f2be82e47f7fde82c9f8e39b" exitCode=0 Jul 28 16:21:15.051466 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.051374 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sswf6" event={"ID":"2a86c24c-ca67-4414-a724-a27350279c9c","Type":"ContainerDied","Data":"761e5e2a23ff336fbdb186093af9a813458a6c63f2be82e47f7fde82c9f8e39b"} Jul 28 16:21:15.065366 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.065313 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-n5jsz" podStartSLOduration=1.565923239 podStartE2EDuration="3.065297909s" totalImagesPullingTime="1.49937467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:15.065108161 +0000 UTC m=+79.923647905" watchObservedRunningTime="2026-07-28 16:21:15.065297909 +0000 UTC m=+79.923837653" Jul 28 16:21:15.105561 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.105509 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-b8f49f5c6-b852l" podStartSLOduration=2.069004849 podStartE2EDuration="3.10549007s" totalImagesPullingTime="1.036485221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:15.083063295 +0000 UTC m=+79.941603038" watchObservedRunningTime="2026-07-28 16:21:15.10549007 +0000 UTC m=+79.964029813" Jul 28 16:21:15.108515 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.108493 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:15.207839 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.207813 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:15.554246 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.554224 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:15.679110 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.679083 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:15.959702 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:15.959644 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:16.056883 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.056855 2574 generic.go:356] "Generic (PLEG): container finished" podID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerID="2ed3b60a6631f750ccd6efea1fc0e15792bd003520e7984f45a46604ff897e4b" exitCode=0 Jul 28 16:21:16.056992 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.056971 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"2ed3b60a6631f750ccd6efea1fc0e15792bd003520e7984f45a46604ff897e4b"} Jul 28 16:21:16.061193 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.061159 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sswf6" event={"ID":"2a86c24c-ca67-4414-a724-a27350279c9c","Type":"ContainerStarted","Data":"035be591a91754f0d332b20c875f2744841f2e7aef37dd5d6919066b8921c12a"} Jul 28 16:21:16.065907 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.061635 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sswf6" event={"ID":"2a86c24c-ca67-4414-a724-a27350279c9c","Type":"ContainerStarted","Data":"fdafe869da7d42ba3432fefcc3909937f89255cabeb075d96a55483b9d78a732"} Jul 28 16:21:16.118229 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.118187 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sswf6" podStartSLOduration=2.8401401440000003 podStartE2EDuration="4.118175319s" totalImagesPullingTime="1.208545279s" totalInitContainerRuntime="69.489896ms" isStatefulPod=true podCreationTimestamp="2026-07-28 16:21:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:16.117933358 +0000 UTC m=+80.976473100" watchObservedRunningTime="2026-07-28 16:21:16.118175319 +0000 UTC m=+80.976715061" Jul 28 16:21:16.466096 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.466063 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:16.566253 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.566212 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:16.662825 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:16.662791 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:17.261283 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.261156 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-599ccbc4fb-xxcfp"] Jul 28 16:21:17.264281 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.264217 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.270820 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.270685 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4chi15eag6d9v\"" Jul 28 16:21:17.271804 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.271781 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 28 16:21:17.271804 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.271795 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7fkhl\"" Jul 28 16:21:17.271966 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.271836 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 28 16:21:17.272217 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.272095 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 28 16:21:17.272217 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.272115 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 28 16:21:17.291636 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.291611 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-599ccbc4fb-xxcfp"] Jul 28 16:21:17.346980 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.346957 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-client-ca-bundle\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.347166 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.347019 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.347166 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.347052 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-audit-log\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.347166 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.347127 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-secret-metrics-server-tls\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.347166 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.347160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-metrics-server-audit-profiles\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.347347 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.347190 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-secret-metrics-server-client-certs\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.347347 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.347242 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9p5z\" (UniqueName: \"kubernetes.io/projected/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-kube-api-access-l9p5z\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.355930 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.355906 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:17.447840 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.447815 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-secret-metrics-server-tls\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.447957 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.447853 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-metrics-server-audit-profiles\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.447957 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.447882 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-secret-metrics-server-client-certs\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.447957 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.447912 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9p5z\" (UniqueName: \"kubernetes.io/projected/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-kube-api-access-l9p5z\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.448088 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.447955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-client-ca-bundle\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.448088 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.448027 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.448088 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.448055 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-audit-log\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.448545 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.448453 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-audit-log\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.448867 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.448843 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.449355 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.449331 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-metrics-server-audit-profiles\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.450900 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.450874 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-secret-metrics-server-tls\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.451018 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.450999 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-secret-metrics-server-client-certs\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.451135 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.451017 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-client-ca-bundle\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.493490 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.493466 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9p5z\" (UniqueName: \"kubernetes.io/projected/03f55659-aa15-4dfe-b2ed-a32b1d131e2c-kube-api-access-l9p5z\") pod \"metrics-server-599ccbc4fb-xxcfp\" (UID: \"03f55659-aa15-4dfe-b2ed-a32b1d131e2c\") " pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.558273 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.558213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:17.576186 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.576160 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:17.659225 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.659194 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:17.700688 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.700661 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-599ccbc4fb-xxcfp"] Jul 28 16:21:17.729036 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.729002 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:17.906012 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.905938 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7564b9d65b-rdllx"] Jul 28 16:21:17.910798 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.910777 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:17.914167 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.913926 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-sxpss\"" Jul 28 16:21:17.914167 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.913983 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 28 16:21:17.914167 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.914026 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 28 16:21:17.914378 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.914264 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 28 16:21:17.914378 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.914312 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 28 16:21:17.914622 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.914595 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 28 16:21:17.919348 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.919326 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 28 16:21:17.928407 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:17.928384 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7564b9d65b-rdllx"] Jul 28 16:21:18.051297 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:18.051268 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03f55659_aa15_4dfe_b2ed_a32b1d131e2c.slice/crio-e5dcdceacdfd73ef731360e61bbe1e15211793baf62ac27eee483fe0f68de760 WatchSource:0}: Error finding container e5dcdceacdfd73ef731360e61bbe1e15211793baf62ac27eee483fe0f68de760: Status 404 returned error can't find the container with id e5dcdceacdfd73ef731360e61bbe1e15211793baf62ac27eee483fe0f68de760 Jul 28 16:21:18.053002 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.052980 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-federate-client-tls\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.053102 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.053010 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-secret-telemeter-client\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.053102 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.053033 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.053102 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.053055 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-serving-certs-ca-bundle\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.053102 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.053078 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-metrics-client-ca\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.053323 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.053130 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-telemeter-client-tls\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.053323 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.053161 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.053323 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.053200 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs4c7\" (UniqueName: \"kubernetes.io/projected/8e90dda2-820d-4702-8416-eecc8ef92112-kube-api-access-fs4c7\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.071615 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.071588 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" event={"ID":"03f55659-aa15-4dfe-b2ed-a32b1d131e2c","Type":"ContainerStarted","Data":"e5dcdceacdfd73ef731360e61bbe1e15211793baf62ac27eee483fe0f68de760"} Jul 28 16:21:18.153786 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.153761 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-secret-telemeter-client\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.153902 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.153803 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.153988 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.153968 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-serving-certs-ca-bundle\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154042 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154002 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-metrics-client-ca\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154099 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154042 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-telemeter-client-tls\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154099 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154061 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154197 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154108 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs4c7\" (UniqueName: \"kubernetes.io/projected/8e90dda2-820d-4702-8416-eecc8ef92112-kube-api-access-fs4c7\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154197 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154166 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-federate-client-tls\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154697 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154671 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-serving-certs-ca-bundle\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154797 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154679 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-metrics-client-ca\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.154797 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.154780 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e90dda2-820d-4702-8416-eecc8ef92112-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.156053 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.155934 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-secret-telemeter-client\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.156592 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.156576 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.156921 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.156899 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-telemeter-client-tls\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.156969 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.156931 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8e90dda2-820d-4702-8416-eecc8ef92112-federate-client-tls\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.163295 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.163276 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs4c7\" (UniqueName: \"kubernetes.io/projected/8e90dda2-820d-4702-8416-eecc8ef92112-kube-api-access-fs4c7\") pod \"telemeter-client-7564b9d65b-rdllx\" (UID: \"8e90dda2-820d-4702-8416-eecc8ef92112\") " pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.223386 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.223368 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" Jul 28 16:21:18.359817 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.359796 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7564b9d65b-rdllx"] Jul 28 16:21:18.361957 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:18.361934 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e90dda2_820d_4702_8416_eecc8ef92112.slice/crio-cc12a49037a884d18e030850ab1883cc0d68e0aff7378b6abfbaa6fad82b7c36 WatchSource:0}: Error finding container cc12a49037a884d18e030850ab1883cc0d68e0aff7378b6abfbaa6fad82b7c36: Status 404 returned error can't find the container with id cc12a49037a884d18e030850ab1883cc0d68e0aff7378b6abfbaa6fad82b7c36 Jul 28 16:21:18.893927 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:18.893891 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:19.077097 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.077058 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" event={"ID":"8e90dda2-820d-4702-8416-eecc8ef92112","Type":"ContainerStarted","Data":"cc12a49037a884d18e030850ab1883cc0d68e0aff7378b6abfbaa6fad82b7c36"} Jul 28 16:21:19.081319 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.081284 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerStarted","Data":"85c2b0f9fa37f0520195e8aee9042470be8f14be6a7bc8d1009d99e609295d35"} Jul 28 16:21:19.081452 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.081365 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerStarted","Data":"d364b3c90a437f9dd9ef261b0820ea9abdc10437e42fe2af13728d0c4219426e"} Jul 28 16:21:19.081452 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.081408 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerStarted","Data":"7561b29eefd2ada27831c49c05e17931347c08d427d2667c3f7562a9289eb070"} Jul 28 16:21:19.081553 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.081449 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerStarted","Data":"b3099aebec1ec024ca04c656907145958039133db4e8894b2f485155566eb7a6"} Jul 28 16:21:19.081553 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.081491 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerStarted","Data":"49aa4585fde20bcb59fcae0b883872713ffd1f8c36f4a4bd532934817698772c"} Jul 28 16:21:19.172345 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.172239 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:19.271320 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:19.271289 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:20.010336 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.010308 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-964c449d9-hrl7v"] Jul 28 16:21:20.013411 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.013392 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.016443 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.016421 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 16:21:20.017572 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.017549 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 16:21:20.017772 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.017751 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ph7rn\"" Jul 28 16:21:20.018029 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.018007 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 16:21:20.018153 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.018076 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 16:21:20.018234 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.018217 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 16:21:20.018306 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.018283 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 16:21:20.018306 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.018305 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 16:21:20.024452 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.024433 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-964c449d9-hrl7v"] Jul 28 16:21:20.089116 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.089085 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerStarted","Data":"b34648377e3c64880a6b80c239e020f37b40ebb7c25c397f6cd40854033ef05a"} Jul 28 16:21:20.091268 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.091246 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" event={"ID":"03f55659-aa15-4dfe-b2ed-a32b1d131e2c","Type":"ContainerStarted","Data":"832bab3eebdc29e5f0ae8ba9f35b7eaac3fa666935d4527b203d9a466227c77c"} Jul 28 16:21:20.117382 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.117340 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.700313036 podStartE2EDuration="7.117326365s" totalImagesPullingTime="4.401165798s" totalInitContainerRuntime="15.847531ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:20.115823479 +0000 UTC m=+84.974363223" watchObservedRunningTime="2026-07-28 16:21:20.117326365 +0000 UTC m=+84.975866102" Jul 28 16:21:20.134927 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.134880 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" podStartSLOduration=1.615397181 podStartE2EDuration="3.134865098s" totalImagesPullingTime="1.519467917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:20.133689211 +0000 UTC m=+84.992228957" watchObservedRunningTime="2026-07-28 16:21:20.134865098 +0000 UTC m=+84.993404842" Jul 28 16:21:20.172097 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.172069 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-config\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.172192 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.172112 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-oauth-serving-cert\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.172192 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.172141 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-oauth-config\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.172323 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.172204 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-serving-cert\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.172323 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.172264 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vksx2\" (UniqueName: \"kubernetes.io/projected/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-kube-api-access-vksx2\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.172419 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.172384 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-service-ca\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.273301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.273226 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vksx2\" (UniqueName: \"kubernetes.io/projected/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-kube-api-access-vksx2\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.273301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.273294 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-service-ca\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.273459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.273350 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-config\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.273459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.273380 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-oauth-serving-cert\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.273459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.273405 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-oauth-config\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.273459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.273440 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-serving-cert\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.274112 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.274082 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-service-ca\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.274274 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.274222 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-config\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.274411 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.274302 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-oauth-serving-cert\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.276277 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.276254 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-serving-cert\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.276393 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.276332 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-oauth-config\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.281914 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.281895 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vksx2\" (UniqueName: \"kubernetes.io/projected/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-kube-api-access-vksx2\") pod \"console-964c449d9-hrl7v\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.324825 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.324808 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:20.469091 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:20.468931 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-964c449d9-hrl7v"] Jul 28 16:21:20.830790 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:20.830756 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f598777_0b4b_4c95_9a16_ed0db2ea59f1.slice/crio-05f87533bb7776bccbdcd02244c80491fac370a855cb7811c62da2a8b57e5545 WatchSource:0}: Error finding container 05f87533bb7776bccbdcd02244c80491fac370a855cb7811c62da2a8b57e5545: Status 404 returned error can't find the container with id 05f87533bb7776bccbdcd02244c80491fac370a855cb7811c62da2a8b57e5545 Jul 28 16:21:21.100970 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:21.100888 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-964c449d9-hrl7v" event={"ID":"7f598777-0b4b-4c95-9a16-ed0db2ea59f1","Type":"ContainerStarted","Data":"05f87533bb7776bccbdcd02244c80491fac370a855cb7811c62da2a8b57e5545"} Jul 28 16:21:21.102890 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:21.102862 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" event={"ID":"8e90dda2-820d-4702-8416-eecc8ef92112","Type":"ContainerStarted","Data":"a0569854f2fb5d37eb9d1685e6ee7528e18d8edf97b62cc38c7f3573b30ee186"} Jul 28 16:21:21.102991 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:21.102928 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" event={"ID":"8e90dda2-820d-4702-8416-eecc8ef92112","Type":"ContainerStarted","Data":"a9878e162cedea05e9e196a9a27dec0f8724c8b261b7012dffbb67aa176d015e"} Jul 28 16:21:21.102991 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:21.102958 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" event={"ID":"8e90dda2-820d-4702-8416-eecc8ef92112","Type":"ContainerStarted","Data":"725ee1f7fe7890f7c1e6a776297bf64e786514e23b058a81d89cd5d29743f2ba"} Jul 28 16:21:21.129243 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:21.129208 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7564b9d65b-rdllx" podStartSLOduration=1.615648104 podStartE2EDuration="4.12919662s" totalImagesPullingTime="2.513548516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:21.127568144 +0000 UTC m=+85.986107888" watchObservedRunningTime="2026-07-28 16:21:21.12919662 +0000 UTC m=+85.987736362" Jul 28 16:21:24.120527 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:24.120489 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:24.224329 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:24.224289 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:25.121821 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:25.121785 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-964c449d9-hrl7v" event={"ID":"7f598777-0b4b-4c95-9a16-ed0db2ea59f1","Type":"ContainerStarted","Data":"5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d"} Jul 28 16:21:25.142318 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:25.142264 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-964c449d9-hrl7v" podStartSLOduration=2.695919215 podStartE2EDuration="6.142248803s" totalImagesPullingTime="3.446329588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:25.141493763 +0000 UTC m=+90.000033509" watchObservedRunningTime="2026-07-28 16:21:25.142248803 +0000 UTC m=+90.000788545" Jul 28 16:21:25.614574 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:25.614543 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:25.716445 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:25.716411 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:25.798502 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:25.798439 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:26.013631 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:26.013597 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:26.113900 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:26.113867 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:26.657070 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:26.657035 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:26.758177 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:26.758140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:29.755243 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.755201 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d65f75675-vn9nf"] Jul 28 16:21:29.776329 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.776304 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d65f75675-vn9nf"] Jul 28 16:21:29.776477 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.776396 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.785335 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.785313 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 16:21:29.849597 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.849571 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-serving-cert\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.849701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.849609 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-service-ca\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.849701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.849627 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-trusted-ca-bundle\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.849701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.849680 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rthrk\" (UniqueName: \"kubernetes.io/projected/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-kube-api-access-rthrk\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.849832 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.849744 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-config\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.849832 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.849810 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-oauth-config\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.849894 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.849852 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-oauth-serving-cert\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950157 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-serving-cert\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950269 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950191 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-service-ca\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950269 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950206 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-trusted-ca-bundle\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950269 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950231 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rthrk\" (UniqueName: \"kubernetes.io/projected/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-kube-api-access-rthrk\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950269 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950258 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-config\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950451 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950303 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-oauth-config\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950451 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950332 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-oauth-serving-cert\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950911 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950892 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-service-ca\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.950994 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.950949 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-config\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.951083 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.951062 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-oauth-serving-cert\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.951162 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.951140 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-trusted-ca-bundle\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.952924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.952898 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-oauth-config\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.953020 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.952968 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-serving-cert\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:29.958141 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:29.958117 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rthrk\" (UniqueName: \"kubernetes.io/projected/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-kube-api-access-rthrk\") pod \"console-6d65f75675-vn9nf\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:30.003372 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:30.003350 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:30.085531 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:30.085483 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:30.213688 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:30.213596 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d65f75675-vn9nf"] Jul 28 16:21:30.216245 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:21:30.216219 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0b37a0_9ea1_466e_b048_79fdc30c78f3.slice/crio-2aa7bce9e92664dc8da85cf74e7a87e26ae4e549091d2bfe8317e5f81d9db7be WatchSource:0}: Error finding container 2aa7bce9e92664dc8da85cf74e7a87e26ae4e549091d2bfe8317e5f81d9db7be: Status 404 returned error can't find the container with id 2aa7bce9e92664dc8da85cf74e7a87e26ae4e549091d2bfe8317e5f81d9db7be Jul 28 16:21:30.325557 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:30.325526 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:30.325691 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:30.325646 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:30.331507 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:30.331481 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:31.150630 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:31.150597 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d65f75675-vn9nf" event={"ID":"5e0b37a0-9ea1-466e-b048-79fdc30c78f3","Type":"ContainerStarted","Data":"fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a"} Jul 28 16:21:31.151073 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:31.150650 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d65f75675-vn9nf" event={"ID":"5e0b37a0-9ea1-466e-b048-79fdc30c78f3","Type":"ContainerStarted","Data":"2aa7bce9e92664dc8da85cf74e7a87e26ae4e549091d2bfe8317e5f81d9db7be"} Jul 28 16:21:31.159296 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:31.159269 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:21:31.169581 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:31.169408 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d65f75675-vn9nf" podStartSLOduration=2.169391578 podStartE2EDuration="2.169391578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:21:31.169241742 +0000 UTC m=+96.027781498" watchObservedRunningTime="2026-07-28 16:21:31.169391578 +0000 UTC m=+96.027931320" Jul 28 16:21:33.475414 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:33.475385 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:33.550614 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:33.550586 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:34.719205 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:34.719164 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:34.820575 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:34.820540 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:37.577339 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:37.577308 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:37.577809 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:37.577372 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:39.018391 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:39.018361 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7hxvb" Jul 28 16:21:40.014307 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.014253 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:40.086462 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.086428 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:40.086833 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.086491 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:40.091145 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.091121 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:40.114473 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.114447 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:40.187320 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.187297 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:21:40.250182 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.250154 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-964c449d9-hrl7v"] Jul 28 16:21:40.470249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.470214 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:40.577976 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:40.577954 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:41.645632 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:41.645598 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:41.749786 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:41.749747 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:42.806179 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:42.806140 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:48.425903 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:48.425861 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:48.536089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:48.536047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:57.573580 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:57.573539 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:21:57.581204 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:57.581179 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:57.584833 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:57.584815 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-599ccbc4fb-xxcfp" Jul 28 16:21:57.674490 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:21:57.674467 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:05.275367 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.275328 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-964c449d9-hrl7v" podUID="7f598777-0b4b-4c95-9a16-ed0db2ea59f1" containerName="console" containerID="cri-o://5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d" gracePeriod=15 Jul 28 16:22:05.512399 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.512378 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:22:05.613585 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.613526 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-service-ca\") pod \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " Jul 28 16:22:05.613585 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.613571 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vksx2\" (UniqueName: \"kubernetes.io/projected/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-kube-api-access-vksx2\") pod \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " Jul 28 16:22:05.613766 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.613603 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-serving-cert\") pod \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " Jul 28 16:22:05.613766 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.613630 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-oauth-config\") pod \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " Jul 28 16:22:05.613766 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.613663 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-config\") pod \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " Jul 28 16:22:05.613766 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.613688 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-oauth-serving-cert\") pod \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\" (UID: \"7f598777-0b4b-4c95-9a16-ed0db2ea59f1\") " Jul 28 16:22:05.614042 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.614011 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-service-ca" (OuterVolumeSpecName: "service-ca") pod "7f598777-0b4b-4c95-9a16-ed0db2ea59f1" (UID: "7f598777-0b4b-4c95-9a16-ed0db2ea59f1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:22:05.614160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.614082 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-config" (OuterVolumeSpecName: "console-config") pod "7f598777-0b4b-4c95-9a16-ed0db2ea59f1" (UID: "7f598777-0b4b-4c95-9a16-ed0db2ea59f1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:22:05.614160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.614114 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7f598777-0b4b-4c95-9a16-ed0db2ea59f1" (UID: "7f598777-0b4b-4c95-9a16-ed0db2ea59f1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:22:05.615852 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.615827 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7f598777-0b4b-4c95-9a16-ed0db2ea59f1" (UID: "7f598777-0b4b-4c95-9a16-ed0db2ea59f1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:05.615933 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.615868 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7f598777-0b4b-4c95-9a16-ed0db2ea59f1" (UID: "7f598777-0b4b-4c95-9a16-ed0db2ea59f1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:05.615933 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.615892 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-kube-api-access-vksx2" (OuterVolumeSpecName: "kube-api-access-vksx2") pod "7f598777-0b4b-4c95-9a16-ed0db2ea59f1" (UID: "7f598777-0b4b-4c95-9a16-ed0db2ea59f1"). InnerVolumeSpecName "kube-api-access-vksx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:22:05.714618 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.714598 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-serving-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:05.714618 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.714618 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-oauth-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:05.714743 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.714627 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-console-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:05.714743 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.714636 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-oauth-serving-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:05.714743 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.714645 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-service-ca\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:05.714743 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:05.714653 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vksx2\" (UniqueName: \"kubernetes.io/projected/7f598777-0b4b-4c95-9a16-ed0db2ea59f1-kube-api-access-vksx2\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:06.286066 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.286038 2574 generic.go:356] "Generic (PLEG): container finished" podID="7f598777-0b4b-4c95-9a16-ed0db2ea59f1" containerID="5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d" exitCode=0 Jul 28 16:22:06.286375 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.286079 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-964c449d9-hrl7v" Jul 28 16:22:06.286375 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.286077 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-964c449d9-hrl7v" event={"ID":"7f598777-0b4b-4c95-9a16-ed0db2ea59f1","Type":"ContainerDied","Data":"5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d"} Jul 28 16:22:06.286375 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.286201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-964c449d9-hrl7v" event={"ID":"7f598777-0b4b-4c95-9a16-ed0db2ea59f1","Type":"ContainerDied","Data":"05f87533bb7776bccbdcd02244c80491fac370a855cb7811c62da2a8b57e5545"} Jul 28 16:22:06.286375 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.286227 2574 scope.go:117] "RemoveContainer" containerID="5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d" Jul 28 16:22:06.296915 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.296896 2574 scope.go:117] "RemoveContainer" containerID="5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d" Jul 28 16:22:06.297211 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:22:06.297187 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d\": container with ID starting with 5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d not found: ID does not exist" containerID="5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d" Jul 28 16:22:06.297295 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.297217 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d"} err="failed to get container status \"5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d\": rpc error: code = NotFound desc = could not find container \"5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d\": container with ID starting with 5e24106b374f5d443c88746cd70528ac6b4131faa5ba7d65dead77db0f439a7d not found: ID does not exist" Jul 28 16:22:06.308475 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.308453 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-964c449d9-hrl7v"] Jul 28 16:22:06.313046 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:06.313027 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-964c449d9-hrl7v"] Jul 28 16:22:07.699633 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:07.699604 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f598777-0b4b-4c95-9a16-ed0db2ea59f1" path="/var/lib/kubelet/pods/7f598777-0b4b-4c95-9a16-ed0db2ea59f1/volumes" Jul 28 16:22:28.824844 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:28.824809 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:28.922002 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:28.921964 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:33.166564 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.166517 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:33.166946 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.166816 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="alertmanager" containerID="cri-o://49aa4585fde20bcb59fcae0b883872713ffd1f8c36f4a4bd532934817698772c" gracePeriod=120 Jul 28 16:22:33.166946 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.166885 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-metric" containerID="cri-o://85c2b0f9fa37f0520195e8aee9042470be8f14be6a7bc8d1009d99e609295d35" gracePeriod=120 Jul 28 16:22:33.167051 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.166914 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-web" containerID="cri-o://7561b29eefd2ada27831c49c05e17931347c08d427d2667c3f7562a9289eb070" gracePeriod=120 Jul 28 16:22:33.167051 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.166986 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="prom-label-proxy" containerID="cri-o://b34648377e3c64880a6b80c239e020f37b40ebb7c25c397f6cd40854033ef05a" gracePeriod=120 Jul 28 16:22:33.167051 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.166925 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="config-reloader" containerID="cri-o://b3099aebec1ec024ca04c656907145958039133db4e8894b2f485155566eb7a6" gracePeriod=120 Jul 28 16:22:33.167201 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.166987 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy" containerID="cri-o://d364b3c90a437f9dd9ef261b0820ea9abdc10437e42fe2af13728d0c4219426e" gracePeriod=120 Jul 28 16:22:33.390605 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390568 2574 generic.go:356] "Generic (PLEG): container finished" podID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerID="b34648377e3c64880a6b80c239e020f37b40ebb7c25c397f6cd40854033ef05a" exitCode=0 Jul 28 16:22:33.390605 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390605 2574 generic.go:356] "Generic (PLEG): container finished" podID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerID="d364b3c90a437f9dd9ef261b0820ea9abdc10437e42fe2af13728d0c4219426e" exitCode=0 Jul 28 16:22:33.390605 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390611 2574 generic.go:356] "Generic (PLEG): container finished" podID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerID="b3099aebec1ec024ca04c656907145958039133db4e8894b2f485155566eb7a6" exitCode=0 Jul 28 16:22:33.390605 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390617 2574 generic.go:356] "Generic (PLEG): container finished" podID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerID="49aa4585fde20bcb59fcae0b883872713ffd1f8c36f4a4bd532934817698772c" exitCode=0 Jul 28 16:22:33.390817 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390638 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"b34648377e3c64880a6b80c239e020f37b40ebb7c25c397f6cd40854033ef05a"} Jul 28 16:22:33.390817 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390698 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"d364b3c90a437f9dd9ef261b0820ea9abdc10437e42fe2af13728d0c4219426e"} Jul 28 16:22:33.390817 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390741 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"b3099aebec1ec024ca04c656907145958039133db4e8894b2f485155566eb7a6"} Jul 28 16:22:33.390817 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:33.390768 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"49aa4585fde20bcb59fcae0b883872713ffd1f8c36f4a4bd532934817698772c"} Jul 28 16:22:34.398427 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.398398 2574 generic.go:356] "Generic (PLEG): container finished" podID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerID="85c2b0f9fa37f0520195e8aee9042470be8f14be6a7bc8d1009d99e609295d35" exitCode=0 Jul 28 16:22:34.398427 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.398427 2574 generic.go:356] "Generic (PLEG): container finished" podID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerID="7561b29eefd2ada27831c49c05e17931347c08d427d2667c3f7562a9289eb070" exitCode=0 Jul 28 16:22:34.398768 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.398456 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"85c2b0f9fa37f0520195e8aee9042470be8f14be6a7bc8d1009d99e609295d35"} Jul 28 16:22:34.398768 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.398517 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"7561b29eefd2ada27831c49c05e17931347c08d427d2667c3f7562a9289eb070"} Jul 28 16:22:34.408731 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.408700 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:34.514980 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.514949 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-tls-assets\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515116 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.514988 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-main-tls\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515116 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515013 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-metrics-client-ca\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515116 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515049 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515116 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515092 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-volume\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515127 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-cluster-tls-config\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515153 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-out\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515206 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svrdt\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-kube-api-access-svrdt\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515232 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-main-db\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515300 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-web\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515538 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515327 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-web-config\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515538 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515356 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515538 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515398 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-trusted-ca-bundle\") pod \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\" (UID: \"c54a2ada-8d99-4719-8c69-4d8c6d1c3961\") " Jul 28 16:22:34.515538 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515411 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:22:34.515760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515621 2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-metrics-client-ca\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.515760 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.515685 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:22:34.516049 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.516027 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:22:34.518252 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.518226 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:22:34.518382 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.518275 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:34.518592 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.518546 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-volume" (OuterVolumeSpecName: "config-volume") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:34.518844 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.518815 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-kube-api-access-svrdt" (OuterVolumeSpecName: "kube-api-access-svrdt") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "kube-api-access-svrdt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:22:34.518954 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.518869 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:34.519215 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.519195 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-out" (OuterVolumeSpecName: "config-out") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:22:34.519285 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.519268 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:34.519694 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.519665 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:34.522306 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.522258 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:34.528054 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.528029 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-web-config" (OuterVolumeSpecName: "web-config") pod "c54a2ada-8d99-4719-8c69-4d8c6d1c3961" (UID: "c54a2ada-8d99-4719-8c69-4d8c6d1c3961"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:22:34.616186 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616139 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616186 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616175 2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-web-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616186 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616185 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616194 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616204 2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-tls-assets\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616212 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-main-tls\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616220 2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616228 2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-volume\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616237 2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-cluster-tls-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616245 2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-config-out\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616254 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svrdt\" (UniqueName: \"kubernetes.io/projected/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-kube-api-access-svrdt\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:34.616310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:34.616261 2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c54a2ada-8d99-4719-8c69-4d8c6d1c3961-alertmanager-main-db\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:22:35.409219 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.409179 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c54a2ada-8d99-4719-8c69-4d8c6d1c3961","Type":"ContainerDied","Data":"5d0ac26b57e914546de1635a55bf3f2f2cded1a945fd7dbb341a672b146b159a"} Jul 28 16:22:35.409591 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.409233 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.409591 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.409302 2574 scope.go:117] "RemoveContainer" containerID="b34648377e3c64880a6b80c239e020f37b40ebb7c25c397f6cd40854033ef05a" Jul 28 16:22:35.416835 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.416812 2574 scope.go:117] "RemoveContainer" containerID="85c2b0f9fa37f0520195e8aee9042470be8f14be6a7bc8d1009d99e609295d35" Jul 28 16:22:35.423625 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.423606 2574 scope.go:117] "RemoveContainer" containerID="d364b3c90a437f9dd9ef261b0820ea9abdc10437e42fe2af13728d0c4219426e" Jul 28 16:22:35.429791 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.429699 2574 scope.go:117] "RemoveContainer" containerID="7561b29eefd2ada27831c49c05e17931347c08d427d2667c3f7562a9289eb070" Jul 28 16:22:35.436202 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.436185 2574 scope.go:117] "RemoveContainer" containerID="b3099aebec1ec024ca04c656907145958039133db4e8894b2f485155566eb7a6" Jul 28 16:22:35.436774 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.436757 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:35.441190 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.441161 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:35.443166 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.443147 2574 scope.go:117] "RemoveContainer" containerID="49aa4585fde20bcb59fcae0b883872713ffd1f8c36f4a4bd532934817698772c" Jul 28 16:22:35.449914 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.449896 2574 scope.go:117] "RemoveContainer" containerID="2ed3b60a6631f750ccd6efea1fc0e15792bd003520e7984f45a46604ff897e4b" Jul 28 16:22:35.475471 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475450 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:35.475740 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475690 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="init-config-reloader" Jul 28 16:22:35.475740 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475703 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="init-config-reloader" Jul 28 16:22:35.475740 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475724 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="alertmanager" Jul 28 16:22:35.475740 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475731 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="alertmanager" Jul 28 16:22:35.475740 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475741 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-web" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475747 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-web" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475755 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f598777-0b4b-4c95-9a16-ed0db2ea59f1" containerName="console" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475761 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f598777-0b4b-4c95-9a16-ed0db2ea59f1" containerName="console" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475767 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475772 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475786 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-metric" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475791 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-metric" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475798 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="config-reloader" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475803 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="config-reloader" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475808 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="prom-label-proxy" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475813 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="prom-label-proxy" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475853 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="alertmanager" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475859 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475865 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f598777-0b4b-4c95-9a16-ed0db2ea59f1" containerName="console" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475872 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="prom-label-proxy" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475878 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-web" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475882 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="kube-rbac-proxy-metric" Jul 28 16:22:35.475924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.475887 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" containerName="config-reloader" Jul 28 16:22:35.479412 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.479397 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.486120 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.486093 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 28 16:22:35.486213 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.486138 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 28 16:22:35.486213 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.486101 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 28 16:22:35.486213 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.486102 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 28 16:22:35.486799 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.486785 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 28 16:22:35.490595 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.490573 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 28 16:22:35.491787 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.491768 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 28 16:22:35.493629 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.493606 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-vb46g\"" Jul 28 16:22:35.493807 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.493790 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 28 16:22:35.494402 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.494383 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:35.495554 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.495536 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 28 16:22:35.623915 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.623885 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.623915 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.623920 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5bgp\" (UniqueName: \"kubernetes.io/projected/ee3db9fe-4bde-44c4-9505-5a185cd365d4-kube-api-access-x5bgp\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624154 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.623941 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3db9fe-4bde-44c4-9505-5a185cd365d4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624154 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.623965 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624154 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624045 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee3db9fe-4bde-44c4-9505-5a185cd365d4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624154 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624097 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee3db9fe-4bde-44c4-9505-5a185cd365d4-config-out\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624154 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624136 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee3db9fe-4bde-44c4-9505-5a185cd365d4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624365 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624158 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-web-config\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624365 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624229 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624365 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624259 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624365 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624282 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624365 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624304 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.624365 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.624319 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3db9fe-4bde-44c4-9505-5a185cd365d4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.699515 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.699486 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54a2ada-8d99-4719-8c69-4d8c6d1c3961" path="/var/lib/kubelet/pods/c54a2ada-8d99-4719-8c69-4d8c6d1c3961/volumes" Jul 28 16:22:35.724959 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.724942 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee3db9fe-4bde-44c4-9505-5a185cd365d4-config-out\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725150 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.724971 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee3db9fe-4bde-44c4-9505-5a185cd365d4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725150 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.724989 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-web-config\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725150 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725010 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725150 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725035 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725150 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725063 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725406 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725212 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725406 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725283 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3db9fe-4bde-44c4-9505-5a185cd365d4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725621 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725598 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725790 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725656 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5bgp\" (UniqueName: \"kubernetes.io/projected/ee3db9fe-4bde-44c4-9505-5a185cd365d4-kube-api-access-x5bgp\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725790 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725692 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3db9fe-4bde-44c4-9505-5a185cd365d4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725790 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725758 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.725960 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.725801 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee3db9fe-4bde-44c4-9505-5a185cd365d4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.726150 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.726130 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ee3db9fe-4bde-44c4-9505-5a185cd365d4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.727297 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.727059 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee3db9fe-4bde-44c4-9505-5a185cd365d4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.728316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.728023 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.728316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.728025 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee3db9fe-4bde-44c4-9505-5a185cd365d4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.728316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.728098 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-web-config\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.728316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.728213 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee3db9fe-4bde-44c4-9505-5a185cd365d4-config-out\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.728316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.728277 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee3db9fe-4bde-44c4-9505-5a185cd365d4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.728316 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.728279 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.728775 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.728598 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-config-volume\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.729460 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.729439 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.729528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.729462 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.730415 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.730398 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee3db9fe-4bde-44c4-9505-5a185cd365d4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.737438 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.737412 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5bgp\" (UniqueName: \"kubernetes.io/projected/ee3db9fe-4bde-44c4-9505-5a185cd365d4-kube-api-access-x5bgp\") pod \"alertmanager-main-0\" (UID: \"ee3db9fe-4bde-44c4-9505-5a185cd365d4\") " pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.788397 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.788374 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 28 16:22:35.927032 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:35.926999 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 28 16:22:35.929788 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:22:35.929765 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee3db9fe_4bde_44c4_9505_5a185cd365d4.slice/crio-2d348d0fee69424f551af74540e2af5882db474e9aa66ea68910b4cd29737b6c WatchSource:0}: Error finding container 2d348d0fee69424f551af74540e2af5882db474e9aa66ea68910b4cd29737b6c: Status 404 returned error can't find the container with id 2d348d0fee69424f551af74540e2af5882db474e9aa66ea68910b4cd29737b6c Jul 28 16:22:36.414144 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:36.414086 2574 generic.go:356] "Generic (PLEG): container finished" podID="ee3db9fe-4bde-44c4-9505-5a185cd365d4" containerID="e174e18a29ed5f2d01fbfb7624012f01b8232accd0c77eef210a0fcb95c59a9a" exitCode=0 Jul 28 16:22:36.414460 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:36.414169 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerDied","Data":"e174e18a29ed5f2d01fbfb7624012f01b8232accd0c77eef210a0fcb95c59a9a"} Jul 28 16:22:36.414460 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:36.414256 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerStarted","Data":"2d348d0fee69424f551af74540e2af5882db474e9aa66ea68910b4cd29737b6c"} Jul 28 16:22:37.422330 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:37.422296 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerStarted","Data":"c33aa05a4940ba7e11871faebe6d608c1c7f3bbc3c321beb5cb296df519f888c"} Jul 28 16:22:37.422657 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:37.422375 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerStarted","Data":"7f57a4dae12168ab5438e9a4b0fe902800f8bc54bedc9507e621a28144a658b8"} Jul 28 16:22:37.422657 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:37.422418 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerStarted","Data":"5597aa06a01c335d508284e81c862c2654864bdf68ad1ba883320c59ed533283"} Jul 28 16:22:37.422657 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:37.422458 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerStarted","Data":"d3c533d7eda3991e03b70b622510b36130bef5b366f13541b299e5f5f20d0416"} Jul 28 16:22:37.422657 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:37.422495 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerStarted","Data":"3a1bd72df5944ae9e12ec43628bcba29ee3cc4eadefc1c5594ba48fe74fb8519"} Jul 28 16:22:37.422657 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:37.422531 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ee3db9fe-4bde-44c4-9505-5a185cd365d4","Type":"ContainerStarted","Data":"c3875ac29f50542723d1dfc5369b084e3544490f1f0f97f3c40356da8a51d319"} Jul 28 16:22:37.467513 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:37.467470 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.4524900450000002 podStartE2EDuration="2.467456039s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.965994ms" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:37.466573561 +0000 UTC m=+162.325113305" watchObservedRunningTime="2026-07-28 16:22:37.467456039 +0000 UTC m=+162.325995781" Jul 28 16:22:38.352418 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.352386 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-65744cc56b-pfdn8"] Jul 28 16:22:38.354526 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.354511 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.377410 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.377389 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65744cc56b-pfdn8"] Jul 28 16:22:38.448876 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.448854 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-console-config\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.449161 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.448891 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-oauth-serving-cert\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.449161 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.448957 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-oauth-config\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.449161 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.448992 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-trusted-ca-bundle\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.449161 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.449016 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-serving-cert\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.449161 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.449034 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-service-ca\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.449161 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.449049 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4l57\" (UniqueName: \"kubernetes.io/projected/476447dd-7140-4ec1-9a93-8092c76d3476-kube-api-access-d4l57\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.457030 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.457009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:38.549490 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.549468 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-trusted-ca-bundle\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.549564 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.549495 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-serving-cert\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.549564 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.549512 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-service-ca\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.549564 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.549526 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4l57\" (UniqueName: \"kubernetes.io/projected/476447dd-7140-4ec1-9a93-8092c76d3476-kube-api-access-d4l57\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.549744 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.549591 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-console-config\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.550359 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.549955 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-oauth-serving-cert\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.550359 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.550045 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-oauth-config\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.550359 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.550339 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-service-ca\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.550580 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.550411 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-console-config\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.550580 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.550459 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-trusted-ca-bundle\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.550681 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.550629 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-oauth-serving-cert\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.552211 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.552190 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-serving-cert\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.552368 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.552352 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-oauth-config\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.563966 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.563948 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4l57\" (UniqueName: \"kubernetes.io/projected/476447dd-7140-4ec1-9a93-8092c76d3476-kube-api-access-d4l57\") pod \"console-65744cc56b-pfdn8\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.569137 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.569118 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:38.664494 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.664450 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:38.791804 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:38.791777 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65744cc56b-pfdn8"] Jul 28 16:22:38.795501 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:22:38.795478 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476447dd_7140_4ec1_9a93_8092c76d3476.slice/crio-98d6685c52fdfc227d1571bcc3d165ac47a40f611dc1f60fb5cdb5fb915e16f2 WatchSource:0}: Error finding container 98d6685c52fdfc227d1571bcc3d165ac47a40f611dc1f60fb5cdb5fb915e16f2: Status 404 returned error can't find the container with id 98d6685c52fdfc227d1571bcc3d165ac47a40f611dc1f60fb5cdb5fb915e16f2 Jul 28 16:22:39.120553 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:39.120526 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:39.221485 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:39.221460 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:39.431276 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:39.431221 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65744cc56b-pfdn8" event={"ID":"476447dd-7140-4ec1-9a93-8092c76d3476","Type":"ContainerStarted","Data":"eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2"} Jul 28 16:22:39.431276 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:39.431268 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65744cc56b-pfdn8" event={"ID":"476447dd-7140-4ec1-9a93-8092c76d3476","Type":"ContainerStarted","Data":"98d6685c52fdfc227d1571bcc3d165ac47a40f611dc1f60fb5cdb5fb915e16f2"} Jul 28 16:22:39.452726 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:39.452670 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-65744cc56b-pfdn8" podStartSLOduration=1.452657166 podStartE2EDuration="1.452657166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:22:39.451990727 +0000 UTC m=+164.310530471" watchObservedRunningTime="2026-07-28 16:22:39.452657166 +0000 UTC m=+164.311196909" Jul 28 16:22:40.729057 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:40.729025 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:40.830975 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:40.830947 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:48.423229 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:48.423157 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:48.524492 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:48.524463 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:48.614035 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:48.614016 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:48.665211 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:48.665189 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:48.665275 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:48.665242 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:48.669651 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:48.669634 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:49.468215 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:49.468187 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:22:49.515354 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:49.515320 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d65f75675-vn9nf"] Jul 28 16:22:49.738787 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:49.738692 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:22:49.825362 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:22:49.825340 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:06.770626 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:06.770594 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:14.534434 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.534377 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d65f75675-vn9nf" podUID="5e0b37a0-9ea1-466e-b048-79fdc30c78f3" containerName="console" containerID="cri-o://fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a" gracePeriod=15 Jul 28 16:23:14.766392 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.766368 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:23:14.809993 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.809932 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rthrk\" (UniqueName: \"kubernetes.io/projected/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-kube-api-access-rthrk\") pod \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " Jul 28 16:23:14.809993 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.809972 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-config\") pod \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " Jul 28 16:23:14.810178 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.810013 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-oauth-config\") pod \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " Jul 28 16:23:14.810178 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.810038 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-serving-cert\") pod \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " Jul 28 16:23:14.810178 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.810092 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-trusted-ca-bundle\") pod \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " Jul 28 16:23:14.810178 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.810114 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-oauth-serving-cert\") pod \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " Jul 28 16:23:14.810178 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.810140 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-service-ca\") pod \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\" (UID: \"5e0b37a0-9ea1-466e-b048-79fdc30c78f3\") " Jul 28 16:23:14.811332 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.811256 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5e0b37a0-9ea1-466e-b048-79fdc30c78f3" (UID: "5e0b37a0-9ea1-466e-b048-79fdc30c78f3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:23:14.811332 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.811271 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-service-ca" (OuterVolumeSpecName: "service-ca") pod "5e0b37a0-9ea1-466e-b048-79fdc30c78f3" (UID: "5e0b37a0-9ea1-466e-b048-79fdc30c78f3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:23:14.812744 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.811514 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-config" (OuterVolumeSpecName: "console-config") pod "5e0b37a0-9ea1-466e-b048-79fdc30c78f3" (UID: "5e0b37a0-9ea1-466e-b048-79fdc30c78f3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:23:14.812744 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.811773 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5e0b37a0-9ea1-466e-b048-79fdc30c78f3" (UID: "5e0b37a0-9ea1-466e-b048-79fdc30c78f3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:23:14.817772 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.817744 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5e0b37a0-9ea1-466e-b048-79fdc30c78f3" (UID: "5e0b37a0-9ea1-466e-b048-79fdc30c78f3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:23:14.817887 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.817764 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5e0b37a0-9ea1-466e-b048-79fdc30c78f3" (UID: "5e0b37a0-9ea1-466e-b048-79fdc30c78f3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:23:14.818253 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.818232 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-kube-api-access-rthrk" (OuterVolumeSpecName: "kube-api-access-rthrk") pod "5e0b37a0-9ea1-466e-b048-79fdc30c78f3" (UID: "5e0b37a0-9ea1-466e-b048-79fdc30c78f3"). InnerVolumeSpecName "kube-api-access-rthrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:23:14.911323 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.911295 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-trusted-ca-bundle\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:23:14.911323 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.911320 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-oauth-serving-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:23:14.911470 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.911330 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-service-ca\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:23:14.911470 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.911339 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rthrk\" (UniqueName: \"kubernetes.io/projected/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-kube-api-access-rthrk\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:23:14.911470 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.911349 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:23:14.911470 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.911358 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-oauth-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:23:14.911470 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:14.911367 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5e0b37a0-9ea1-466e-b048-79fdc30c78f3-console-serving-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:23:15.561566 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.561529 2574 generic.go:356] "Generic (PLEG): container finished" podID="5e0b37a0-9ea1-466e-b048-79fdc30c78f3" containerID="fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a" exitCode=0 Jul 28 16:23:15.562090 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.561590 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d65f75675-vn9nf" Jul 28 16:23:15.562090 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.561605 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d65f75675-vn9nf" event={"ID":"5e0b37a0-9ea1-466e-b048-79fdc30c78f3","Type":"ContainerDied","Data":"fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a"} Jul 28 16:23:15.562090 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.561680 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d65f75675-vn9nf" event={"ID":"5e0b37a0-9ea1-466e-b048-79fdc30c78f3","Type":"ContainerDied","Data":"2aa7bce9e92664dc8da85cf74e7a87e26ae4e549091d2bfe8317e5f81d9db7be"} Jul 28 16:23:15.562090 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.561701 2574 scope.go:117] "RemoveContainer" containerID="fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a" Jul 28 16:23:15.570159 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.570137 2574 scope.go:117] "RemoveContainer" containerID="fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a" Jul 28 16:23:15.570424 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:23:15.570405 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a\": container with ID starting with fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a not found: ID does not exist" containerID="fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a" Jul 28 16:23:15.570523 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.570434 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a"} err="failed to get container status \"fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a\": rpc error: code = NotFound desc = could not find container \"fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a\": container with ID starting with fb0903b763e588d299d856047c9c1249013b756b539b933cb40135e35b51fa4a not found: ID does not exist" Jul 28 16:23:15.582735 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.582694 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d65f75675-vn9nf"] Jul 28 16:23:15.587028 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.587002 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d65f75675-vn9nf"] Jul 28 16:23:15.699328 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:15.699294 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e0b37a0-9ea1-466e-b048-79fdc30c78f3" path="/var/lib/kubelet/pods/5e0b37a0-9ea1-466e-b048-79fdc30c78f3/volumes" Jul 28 16:23:40.405138 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:40.405107 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:40.506310 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:40.506283 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:41.008870 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:41.008838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:41.119653 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:41.119620 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:23:42.214762 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.214730 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn"] Jul 28 16:23:42.215126 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.215009 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e0b37a0-9ea1-466e-b048-79fdc30c78f3" containerName="console" Jul 28 16:23:42.215126 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.215019 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e0b37a0-9ea1-466e-b048-79fdc30c78f3" containerName="console" Jul 28 16:23:42.215126 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.215067 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e0b37a0-9ea1-466e-b048-79fdc30c78f3" containerName="console" Jul 28 16:23:42.218041 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.218024 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.221834 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.221806 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 16:23:42.221961 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.221913 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2xdtf\"" Jul 28 16:23:42.223011 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.222983 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 16:23:42.230658 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.230632 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn"] Jul 28 16:23:42.319704 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.319676 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.319818 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.319753 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r5tx\" (UniqueName: \"kubernetes.io/projected/99c11c6f-1053-43d1-9055-817da26ae90f-kube-api-access-4r5tx\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.319818 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.319778 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.420451 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.420430 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.420509 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.420481 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r5tx\" (UniqueName: \"kubernetes.io/projected/99c11c6f-1053-43d1-9055-817da26ae90f-kube-api-access-4r5tx\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.420509 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.420504 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.420867 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.420851 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.420918 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.420870 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.431856 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.431830 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r5tx\" (UniqueName: \"kubernetes.io/projected/99c11c6f-1053-43d1-9055-817da26ae90f-kube-api-access-4r5tx\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.527060 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.527005 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:23:42.661989 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:42.661963 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn"] Jul 28 16:23:42.666074 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:23:42.666050 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99c11c6f_1053_43d1_9055_817da26ae90f.slice/crio-cc1c3aeec4b9f92be9565e827b5f728f51c0144857546f66dca2333ec2a2a28d WatchSource:0}: Error finding container cc1c3aeec4b9f92be9565e827b5f728f51c0144857546f66dca2333ec2a2a28d: Status 404 returned error can't find the container with id cc1c3aeec4b9f92be9565e827b5f728f51c0144857546f66dca2333ec2a2a28d Jul 28 16:23:43.670969 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:43.670937 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" event={"ID":"99c11c6f-1053-43d1-9055-817da26ae90f","Type":"ContainerStarted","Data":"cc1c3aeec4b9f92be9565e827b5f728f51c0144857546f66dca2333ec2a2a28d"} Jul 28 16:23:49.695669 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:49.695632 2574 generic.go:356] "Generic (PLEG): container finished" podID="99c11c6f-1053-43d1-9055-817da26ae90f" containerID="bc3432b24a9b300c3893dd0ac39b6b7e7011506b3dc8dfbc557d8c6ca82e195b" exitCode=0 Jul 28 16:23:49.701642 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:49.701610 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" event={"ID":"99c11c6f-1053-43d1-9055-817da26ae90f","Type":"ContainerDied","Data":"bc3432b24a9b300c3893dd0ac39b6b7e7011506b3dc8dfbc557d8c6ca82e195b"} Jul 28 16:23:52.707019 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:52.706931 2574 generic.go:356] "Generic (PLEG): container finished" podID="99c11c6f-1053-43d1-9055-817da26ae90f" containerID="05fdbc080482f1bf57bb4d217f322e42bcf0370ceee02920550609e69efad837" exitCode=0 Jul 28 16:23:52.707019 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:52.706977 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" event={"ID":"99c11c6f-1053-43d1-9055-817da26ae90f","Type":"ContainerDied","Data":"05fdbc080482f1bf57bb4d217f322e42bcf0370ceee02920550609e69efad837"} Jul 28 16:23:59.734459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:59.734426 2574 generic.go:356] "Generic (PLEG): container finished" podID="99c11c6f-1053-43d1-9055-817da26ae90f" containerID="8246e755c2bea74252902606aadb595d0ca0aba17f6b7b7b903f220bff1c3507" exitCode=0 Jul 28 16:23:59.734459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:23:59.734463 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" event={"ID":"99c11c6f-1053-43d1-9055-817da26ae90f","Type":"ContainerDied","Data":"8246e755c2bea74252902606aadb595d0ca0aba17f6b7b7b903f220bff1c3507"} Jul 28 16:24:00.404366 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.404328 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:00.504725 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.504675 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:00.858021 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.857991 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:24:00.862788 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.862770 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r5tx\" (UniqueName: \"kubernetes.io/projected/99c11c6f-1053-43d1-9055-817da26ae90f-kube-api-access-4r5tx\") pod \"99c11c6f-1053-43d1-9055-817da26ae90f\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " Jul 28 16:24:00.862892 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.862820 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-bundle\") pod \"99c11c6f-1053-43d1-9055-817da26ae90f\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " Jul 28 16:24:00.862892 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.862844 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-util\") pod \"99c11c6f-1053-43d1-9055-817da26ae90f\" (UID: \"99c11c6f-1053-43d1-9055-817da26ae90f\") " Jul 28 16:24:00.863553 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.863528 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-bundle" (OuterVolumeSpecName: "bundle") pod "99c11c6f-1053-43d1-9055-817da26ae90f" (UID: "99c11c6f-1053-43d1-9055-817da26ae90f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:24:00.864905 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.864882 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99c11c6f-1053-43d1-9055-817da26ae90f-kube-api-access-4r5tx" (OuterVolumeSpecName: "kube-api-access-4r5tx") pod "99c11c6f-1053-43d1-9055-817da26ae90f" (UID: "99c11c6f-1053-43d1-9055-817da26ae90f"). InnerVolumeSpecName "kube-api-access-4r5tx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:24:00.866787 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.866764 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-util" (OuterVolumeSpecName: "util") pod "99c11c6f-1053-43d1-9055-817da26ae90f" (UID: "99c11c6f-1053-43d1-9055-817da26ae90f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:24:00.963319 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.963285 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r5tx\" (UniqueName: \"kubernetes.io/projected/99c11c6f-1053-43d1-9055-817da26ae90f-kube-api-access-4r5tx\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:24:00.963319 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.963312 2574 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-bundle\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:24:00.963319 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:00.963321 2574 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99c11c6f-1053-43d1-9055-817da26ae90f-util\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:24:01.742900 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:01.742870 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" event={"ID":"99c11c6f-1053-43d1-9055-817da26ae90f","Type":"ContainerDied","Data":"cc1c3aeec4b9f92be9565e827b5f728f51c0144857546f66dca2333ec2a2a28d"} Jul 28 16:24:01.743025 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:01.742912 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5xmwwn" Jul 28 16:24:01.743025 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:01.742914 2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc1c3aeec4b9f92be9565e827b5f728f51c0144857546f66dca2333ec2a2a28d" Jul 28 16:24:03.779299 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:03.779266 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:03.922692 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:03.922653 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:05.854384 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854350 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8"] Jul 28 16:24:05.854765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854677 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c11c6f-1053-43d1-9055-817da26ae90f" containerName="util" Jul 28 16:24:05.854765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854689 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c11c6f-1053-43d1-9055-817da26ae90f" containerName="util" Jul 28 16:24:05.854765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854696 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c11c6f-1053-43d1-9055-817da26ae90f" containerName="pull" Jul 28 16:24:05.854765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854702 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c11c6f-1053-43d1-9055-817da26ae90f" containerName="pull" Jul 28 16:24:05.854765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854723 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99c11c6f-1053-43d1-9055-817da26ae90f" containerName="extract" Jul 28 16:24:05.854765 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854729 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="99c11c6f-1053-43d1-9055-817da26ae90f" containerName="extract" Jul 28 16:24:05.854948 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.854772 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="99c11c6f-1053-43d1-9055-817da26ae90f" containerName="extract" Jul 28 16:24:05.887092 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.887063 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8"] Jul 28 16:24:05.887318 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.887167 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:05.891334 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.891309 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 28 16:24:05.892164 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.892143 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-d9sxx\"" Jul 28 16:24:05.892287 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.892169 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 28 16:24:05.893467 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.893443 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 28 16:24:05.897525 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.897504 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk25s\" (UniqueName: \"kubernetes.io/projected/2198a64d-c4ae-409e-99b9-7d4f026f4d40-kube-api-access-wk25s\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8\" (UID: \"2198a64d-c4ae-409e-99b9-7d4f026f4d40\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:05.897606 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.897560 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2198a64d-c4ae-409e-99b9-7d4f026f4d40-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8\" (UID: \"2198a64d-c4ae-409e-99b9-7d4f026f4d40\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:05.998479 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.998446 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk25s\" (UniqueName: \"kubernetes.io/projected/2198a64d-c4ae-409e-99b9-7d4f026f4d40-kube-api-access-wk25s\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8\" (UID: \"2198a64d-c4ae-409e-99b9-7d4f026f4d40\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:05.998636 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:05.998500 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2198a64d-c4ae-409e-99b9-7d4f026f4d40-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8\" (UID: \"2198a64d-c4ae-409e-99b9-7d4f026f4d40\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:06.000952 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:06.000919 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2198a64d-c4ae-409e-99b9-7d4f026f4d40-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8\" (UID: \"2198a64d-c4ae-409e-99b9-7d4f026f4d40\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:06.011335 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:06.011311 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk25s\" (UniqueName: \"kubernetes.io/projected/2198a64d-c4ae-409e-99b9-7d4f026f4d40-kube-api-access-wk25s\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8\" (UID: \"2198a64d-c4ae-409e-99b9-7d4f026f4d40\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:06.199347 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:06.199260 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:06.361301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:06.361260 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8"] Jul 28 16:24:06.363828 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:24:06.363802 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2198a64d_c4ae_409e_99b9_7d4f026f4d40.slice/crio-cc7a5aca50474810848766245d5e2bb7d1f9283a2d9a6bf6914869fde2746c32 WatchSource:0}: Error finding container cc7a5aca50474810848766245d5e2bb7d1f9283a2d9a6bf6914869fde2746c32: Status 404 returned error can't find the container with id cc7a5aca50474810848766245d5e2bb7d1f9283a2d9a6bf6914869fde2746c32 Jul 28 16:24:06.761299 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:06.761266 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" event={"ID":"2198a64d-c4ae-409e-99b9-7d4f026f4d40","Type":"ContainerStarted","Data":"cc7a5aca50474810848766245d5e2bb7d1f9283a2d9a6bf6914869fde2746c32"} Jul 28 16:24:09.775848 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:09.775812 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" event={"ID":"2198a64d-c4ae-409e-99b9-7d4f026f4d40","Type":"ContainerStarted","Data":"09b2815c8453ce37a3428c9c9a0cbf37bdd2c9a1162760e8efd5e4ff3570aab5"} Jul 28 16:24:09.776122 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:09.775948 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:09.814794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:09.814738 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" podStartSLOduration=1.504028598 podStartE2EDuration="4.814707809s" totalImagesPullingTime="3.310679211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:09.813585635 +0000 UTC m=+254.672125377" watchObservedRunningTime="2026-07-28 16:24:09.814707809 +0000 UTC m=+254.673247551" Jul 28 16:24:10.213158 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.213128 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:10.595899 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.595827 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-8r8gf"] Jul 28 16:24:10.598116 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.598093 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.599183 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.599158 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:10.601164 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.601143 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-t9b4t\"" Jul 28 16:24:10.601402 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.601388 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 28 16:24:10.601450 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.601392 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 28 16:24:10.622476 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.622455 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-8r8gf"] Jul 28 16:24:10.634928 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.634908 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.635007 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.634935 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t876h\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-kube-api-access-t876h\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.635007 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.634961 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/23aa6944-62b9-4443-b12e-a6231adf4087-cabundle0\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.735252 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.735228 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.735351 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.735253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t876h\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-kube-api-access-t876h\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.735351 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.735288 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/23aa6944-62b9-4443-b12e-a6231adf4087-cabundle0\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.735421 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:10.735381 2574 secret.go:281] references non-existent secret key: ca.crt Jul 28 16:24:10.735421 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:10.735401 2574 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 28 16:24:10.735421 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:10.735413 2574 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-8r8gf: references non-existent secret key: ca.crt Jul 28 16:24:10.735510 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:10.735475 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates podName:23aa6944-62b9-4443-b12e-a6231adf4087 nodeName:}" failed. No retries permitted until 2026-07-28 16:24:11.23545605 +0000 UTC m=+256.093995784 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates") pod "keda-operator-77f648d694-8r8gf" (UID: "23aa6944-62b9-4443-b12e-a6231adf4087") : references non-existent secret key: ca.crt Jul 28 16:24:10.735919 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.735904 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/23aa6944-62b9-4443-b12e-a6231adf4087-cabundle0\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.759168 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.759141 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t876h\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-kube-api-access-t876h\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:10.801239 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:10.801205 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:11.239194 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.239165 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:11.239350 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:11.239319 2574 secret.go:281] references non-existent secret key: ca.crt Jul 28 16:24:11.239350 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:11.239341 2574 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 28 16:24:11.239460 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:11.239353 2574 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-8r8gf: references non-existent secret key: ca.crt Jul 28 16:24:11.239460 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:11.239415 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates podName:23aa6944-62b9-4443-b12e-a6231adf4087 nodeName:}" failed. No retries permitted until 2026-07-28 16:24:12.239398071 +0000 UTC m=+257.097937806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates") pod "keda-operator-77f648d694-8r8gf" (UID: "23aa6944-62b9-4443-b12e-a6231adf4087") : references non-existent secret key: ca.crt Jul 28 16:24:11.445671 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.445643 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:11.700979 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.700906 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-shvr4"] Jul 28 16:24:11.703094 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.703078 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.708078 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.708057 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 28 16:24:11.731248 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.731224 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-shvr4"] Jul 28 16:24:11.743984 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.743960 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgcbr\" (UniqueName: \"kubernetes.io/projected/7994b64d-26e3-4f1e-998c-9a1822fc3684-kube-api-access-qgcbr\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.744129 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.744090 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7994b64d-26e3-4f1e-998c-9a1822fc3684-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.744291 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.744269 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7994b64d-26e3-4f1e-998c-9a1822fc3684-certificates\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.845305 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.845263 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7994b64d-26e3-4f1e-998c-9a1822fc3684-certificates\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.845701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.845324 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgcbr\" (UniqueName: \"kubernetes.io/projected/7994b64d-26e3-4f1e-998c-9a1822fc3684-kube-api-access-qgcbr\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.845701 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.845375 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7994b64d-26e3-4f1e-998c-9a1822fc3684-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.845861 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.845816 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/7994b64d-26e3-4f1e-998c-9a1822fc3684-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.847645 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.847625 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7994b64d-26e3-4f1e-998c-9a1822fc3684-certificates\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.855258 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.855240 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgcbr\" (UniqueName: \"kubernetes.io/projected/7994b64d-26e3-4f1e-998c-9a1822fc3684-kube-api-access-qgcbr\") pod \"keda-metrics-apiserver-55557df8c-shvr4\" (UID: \"7994b64d-26e3-4f1e-998c-9a1822fc3684\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:11.858069 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.858047 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:11.973208 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.973183 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-vjlqh"] Jul 28 16:24:11.975391 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.975377 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:11.978486 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:11.978466 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 28 16:24:12.001069 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.001044 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-vjlqh"] Jul 28 16:24:12.013693 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.013671 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:12.014049 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.014030 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:12.046879 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.046843 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-certificates\") pod \"keda-admission-56f9798bb4-vjlqh\" (UID: \"ecfc6d2a-e823-461f-96da-e592f35ee7f4\") " pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.047019 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.046899 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwdwt\" (UniqueName: \"kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-kube-api-access-fwdwt\") pod \"keda-admission-56f9798bb4-vjlqh\" (UID: \"ecfc6d2a-e823-461f-96da-e592f35ee7f4\") " pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.147577 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.147547 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-certificates\") pod \"keda-admission-56f9798bb4-vjlqh\" (UID: \"ecfc6d2a-e823-461f-96da-e592f35ee7f4\") " pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.147705 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.147603 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwdwt\" (UniqueName: \"kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-kube-api-access-fwdwt\") pod \"keda-admission-56f9798bb4-vjlqh\" (UID: \"ecfc6d2a-e823-461f-96da-e592f35ee7f4\") " pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.147783 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:12.147726 2574 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found Jul 28 16:24:12.147783 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:12.147748 2574 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-vjlqh: secret "keda-admission-webhooks-certs" not found Jul 28 16:24:12.147854 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:24:12.147807 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-certificates podName:ecfc6d2a-e823-461f-96da-e592f35ee7f4 nodeName:}" failed. No retries permitted until 2026-07-28 16:24:12.647786043 +0000 UTC m=+257.506325771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-certificates") pod "keda-admission-56f9798bb4-vjlqh" (UID: "ecfc6d2a-e823-461f-96da-e592f35ee7f4") : secret "keda-admission-webhooks-certs" not found Jul 28 16:24:12.171335 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.171311 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwdwt\" (UniqueName: \"kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-kube-api-access-fwdwt\") pod \"keda-admission-56f9798bb4-vjlqh\" (UID: \"ecfc6d2a-e823-461f-96da-e592f35ee7f4\") " pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.198413 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.198390 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-shvr4"] Jul 28 16:24:12.201455 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:24:12.201430 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7994b64d_26e3_4f1e_998c_9a1822fc3684.slice/crio-9b28ff327034ab76249ba6f2ff745ffecbf8956f315b15c3f0b6930a7ab75888 WatchSource:0}: Error finding container 9b28ff327034ab76249ba6f2ff745ffecbf8956f315b15c3f0b6930a7ab75888: Status 404 returned error can't find the container with id 9b28ff327034ab76249ba6f2ff745ffecbf8956f315b15c3f0b6930a7ab75888 Jul 28 16:24:12.230816 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.230763 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:12.248070 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.248041 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:12.250233 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.250210 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/23aa6944-62b9-4443-b12e-a6231adf4087-certificates\") pod \"keda-operator-77f648d694-8r8gf\" (UID: \"23aa6944-62b9-4443-b12e-a6231adf4087\") " pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:12.408528 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.408506 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:12.547255 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.547228 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-8r8gf"] Jul 28 16:24:12.554854 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:24:12.554826 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23aa6944_62b9_4443_b12e_a6231adf4087.slice/crio-e63b25019adaa0dec34ba5fe7f456f69494db9af68efd1fe46a36defd8433715 WatchSource:0}: Error finding container e63b25019adaa0dec34ba5fe7f456f69494db9af68efd1fe46a36defd8433715: Status 404 returned error can't find the container with id e63b25019adaa0dec34ba5fe7f456f69494db9af68efd1fe46a36defd8433715 Jul 28 16:24:12.650863 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.650841 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-certificates\") pod \"keda-admission-56f9798bb4-vjlqh\" (UID: \"ecfc6d2a-e823-461f-96da-e592f35ee7f4\") " pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.653038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.653014 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ecfc6d2a-e823-461f-96da-e592f35ee7f4-certificates\") pod \"keda-admission-56f9798bb4-vjlqh\" (UID: \"ecfc6d2a-e823-461f-96da-e592f35ee7f4\") " pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.788393 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.788329 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-8r8gf" event={"ID":"23aa6944-62b9-4443-b12e-a6231adf4087","Type":"ContainerStarted","Data":"e63b25019adaa0dec34ba5fe7f456f69494db9af68efd1fe46a36defd8433715"} Jul 28 16:24:12.789355 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.789335 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" event={"ID":"7994b64d-26e3-4f1e-998c-9a1822fc3684","Type":"ContainerStarted","Data":"9b28ff327034ab76249ba6f2ff745ffecbf8956f315b15c3f0b6930a7ab75888"} Jul 28 16:24:12.884888 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.884862 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:12.889004 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:12.888981 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:13.055065 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:13.055024 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-vjlqh"] Jul 28 16:24:13.057722 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:24:13.057669 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecfc6d2a_e823_461f_96da_e592f35ee7f4.slice/crio-c462a064fa71279eae3df882f54bc1240e8b0b1f36957905b1b2ffd5a71e9cbd WatchSource:0}: Error finding container c462a064fa71279eae3df882f54bc1240e8b0b1f36957905b1b2ffd5a71e9cbd: Status 404 returned error can't find the container with id c462a064fa71279eae3df882f54bc1240e8b0b1f36957905b1b2ffd5a71e9cbd Jul 28 16:24:13.130248 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:13.130221 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:13.487317 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:13.487283 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:13.796587 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:13.796504 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" event={"ID":"ecfc6d2a-e823-461f-96da-e592f35ee7f4","Type":"ContainerStarted","Data":"c462a064fa71279eae3df882f54bc1240e8b0b1f36957905b1b2ffd5a71e9cbd"} Jul 28 16:24:16.811961 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.811923 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" event={"ID":"7994b64d-26e3-4f1e-998c-9a1822fc3684","Type":"ContainerStarted","Data":"2ba9ce111d6904b3e189991758fc314ce22304aae225242d33ad026b519b4a4a"} Jul 28 16:24:16.812371 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.812025 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:16.813428 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.813395 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-8r8gf" event={"ID":"23aa6944-62b9-4443-b12e-a6231adf4087","Type":"ContainerStarted","Data":"4002fe1be34d358c7d33671bf7803d58277e803e9ac21310a21922fd25da1dd4"} Jul 28 16:24:16.813554 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.813494 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:16.814752 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.814733 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" event={"ID":"ecfc6d2a-e823-461f-96da-e592f35ee7f4","Type":"ContainerStarted","Data":"da8f32a92a1a21dc680bb3bc1bfb59c8b6887565ef3a63599d6cfc4e86c0f8b9"} Jul 28 16:24:16.814849 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.814835 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:16.835143 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.835091 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" podStartSLOduration=1.684912056 podStartE2EDuration="5.835074584s" totalImagesPullingTime="4.150162528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:16.834227052 +0000 UTC m=+261.692766795" watchObservedRunningTime="2026-07-28 16:24:16.835074584 +0000 UTC m=+261.693614328" Jul 28 16:24:16.855570 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.855526 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" podStartSLOduration=2.558155869 podStartE2EDuration="5.855513966s" totalImagesPullingTime="3.297358097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:16.853889389 +0000 UTC m=+261.712429132" watchObservedRunningTime="2026-07-28 16:24:16.855513966 +0000 UTC m=+261.714053708" Jul 28 16:24:16.877221 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:16.877180 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-8r8gf" podStartSLOduration=3.079554521 podStartE2EDuration="6.877168241s" totalImagesPullingTime="3.79761372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:24:16.8763827 +0000 UTC m=+261.734922444" watchObservedRunningTime="2026-07-28 16:24:16.877168241 +0000 UTC m=+261.735707983" Jul 28 16:24:27.822609 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:27.822560 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-shvr4" Jul 28 16:24:30.781260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:30.781225 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lwvb8" Jul 28 16:24:37.821085 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:37.821050 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-8r8gf" Jul 28 16:24:37.821531 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:37.821482 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-vjlqh" Jul 28 16:24:55.533589 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:55.533548 2574 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 16:24:56.613875 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:56.613838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:24:58.149318 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:24:58.149279 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:07.788440 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:07.788375 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:31.909788 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.909745 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:31.979137 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.979103 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-6vs58"] Jul 28 16:25:31.981337 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.981316 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:31.984197 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.984175 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 16:25:31.984303 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.984225 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 16:25:31.984303 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.984235 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 28 16:25:31.985242 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.985225 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-jlgxf\"" Jul 28 16:25:31.992345 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.992325 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-4ldgp"] Jul 28 16:25:31.994401 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.994387 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:31.997023 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.996998 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-kvvmv\"" Jul 28 16:25:31.997107 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:31.997073 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 28 16:25:32.002399 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.002378 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-6vs58"] Jul 28 16:25:32.008252 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.008228 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:32.013198 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.013176 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-4ldgp"] Jul 28 16:25:32.023481 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.023448 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186b27a3-b7f1-4718-8f5b-938407afdd93-cert\") pod \"kserve-controller-manager-85d8b8ddfc-6vs58\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:32.023574 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.023499 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g2n7\" (UniqueName: \"kubernetes.io/projected/186b27a3-b7f1-4718-8f5b-938407afdd93-kube-api-access-7g2n7\") pod \"kserve-controller-manager-85d8b8ddfc-6vs58\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:32.023574 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.023565 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5-cert\") pod \"llmisvc-controller-manager-84fc959947-4ldgp\" (UID: \"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5\") " pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:32.023667 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.023585 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xbk\" (UniqueName: \"kubernetes.io/projected/7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5-kube-api-access-24xbk\") pod \"llmisvc-controller-manager-84fc959947-4ldgp\" (UID: \"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5\") " pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:32.106240 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.106212 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:32.124278 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.124253 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186b27a3-b7f1-4718-8f5b-938407afdd93-cert\") pod \"kserve-controller-manager-85d8b8ddfc-6vs58\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:32.124376 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.124297 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g2n7\" (UniqueName: \"kubernetes.io/projected/186b27a3-b7f1-4718-8f5b-938407afdd93-kube-api-access-7g2n7\") pod \"kserve-controller-manager-85d8b8ddfc-6vs58\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:32.124376 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.124359 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5-cert\") pod \"llmisvc-controller-manager-84fc959947-4ldgp\" (UID: \"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5\") " pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:32.124546 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.124390 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24xbk\" (UniqueName: \"kubernetes.io/projected/7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5-kube-api-access-24xbk\") pod \"llmisvc-controller-manager-84fc959947-4ldgp\" (UID: \"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5\") " pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:32.126656 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.126630 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186b27a3-b7f1-4718-8f5b-938407afdd93-cert\") pod \"kserve-controller-manager-85d8b8ddfc-6vs58\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:32.126778 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.126705 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5-cert\") pod \"llmisvc-controller-manager-84fc959947-4ldgp\" (UID: \"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5\") " pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:32.134104 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.134084 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xbk\" (UniqueName: \"kubernetes.io/projected/7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5-kube-api-access-24xbk\") pod \"llmisvc-controller-manager-84fc959947-4ldgp\" (UID: \"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5\") " pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:32.134445 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.134424 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g2n7\" (UniqueName: \"kubernetes.io/projected/186b27a3-b7f1-4718-8f5b-938407afdd93-kube-api-access-7g2n7\") pod \"kserve-controller-manager-85d8b8ddfc-6vs58\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:32.207882 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.207829 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:32.292124 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.292097 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:32.303089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.303059 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:32.433889 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.433867 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-6vs58"] Jul 28 16:25:32.436589 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:25:32.436565 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod186b27a3_b7f1_4718_8f5b_938407afdd93.slice/crio-22ecd73eb7e50c40774d07fe4fc190a4a25da90059c484e9a5dfcfaba95c97e8 WatchSource:0}: Error finding container 22ecd73eb7e50c40774d07fe4fc190a4a25da90059c484e9a5dfcfaba95c97e8: Status 404 returned error can't find the container with id 22ecd73eb7e50c40774d07fe4fc190a4a25da90059c484e9a5dfcfaba95c97e8 Jul 28 16:25:32.437882 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.437865 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:25:32.449950 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.449931 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-4ldgp"] Jul 28 16:25:32.452023 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:25:32.452002 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7e5c7dc5_9fc0_4e2f_b9b0_26137b3348d5.slice/crio-7bc0dfead4f7788e7435df10d68cfeb2a32142dcebba1334b9af53799673cb3b WatchSource:0}: Error finding container 7bc0dfead4f7788e7435df10d68cfeb2a32142dcebba1334b9af53799673cb3b: Status 404 returned error can't find the container with id 7bc0dfead4f7788e7435df10d68cfeb2a32142dcebba1334b9af53799673cb3b Jul 28 16:25:32.585569 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.585542 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:32.686840 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.686816 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:32.748696 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:32.748670 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:25:33.142778 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:33.142745 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" event={"ID":"186b27a3-b7f1-4718-8f5b-938407afdd93","Type":"ContainerStarted","Data":"22ecd73eb7e50c40774d07fe4fc190a4a25da90059c484e9a5dfcfaba95c97e8"} Jul 28 16:25:33.144068 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:33.144044 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" event={"ID":"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5","Type":"ContainerStarted","Data":"7bc0dfead4f7788e7435df10d68cfeb2a32142dcebba1334b9af53799673cb3b"} Jul 28 16:25:36.160452 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:36.160355 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" event={"ID":"7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5","Type":"ContainerStarted","Data":"4b778e144e2ef0b7adc3bce57626a0b6358d0f300c745ee5393713d56271c294"} Jul 28 16:25:36.160882 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:36.160448 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:25:36.161846 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:36.161814 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" event={"ID":"186b27a3-b7f1-4718-8f5b-938407afdd93","Type":"ContainerStarted","Data":"f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94"} Jul 28 16:25:36.161972 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:36.161959 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:25:36.184224 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:25:36.184181 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" podStartSLOduration=1.8121534719999999 podStartE2EDuration="5.184169412s" totalImagesPullingTime="3.37201594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:25:36.183924808 +0000 UTC m=+341.042464607" watchObservedRunningTime="2026-07-28 16:25:36.184169412 +0000 UTC m=+341.042709154" Jul 28 16:26:07.168393 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:07.168317 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-84fc959947-4ldgp" Jul 28 16:26:07.171295 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:07.171275 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:26:07.190776 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:07.190729 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" podStartSLOduration=32.852243911 podStartE2EDuration="36.190700453s" totalImagesPullingTime="3.338456542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:25:36.20986315 +0000 UTC m=+341.068402893" watchObservedRunningTime="2026-07-28 16:26:07.190700453 +0000 UTC m=+372.049240225" Jul 28 16:26:15.677595 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.677563 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-6vs58"] Jul 28 16:26:15.678083 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.677796 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" podUID="186b27a3-b7f1-4718-8f5b-938407afdd93" containerName="manager" containerID="cri-o://f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94" gracePeriod=10 Jul 28 16:26:15.702771 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.702695 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-hk4vr"] Jul 28 16:26:15.705055 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.705037 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:15.720678 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.720656 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-hk4vr"] Jul 28 16:26:15.860279 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.860251 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkrwd\" (UniqueName: \"kubernetes.io/projected/13d4cc11-ed3f-4890-ada9-0f1100dd2de3-kube-api-access-nkrwd\") pod \"kserve-controller-manager-85d8b8ddfc-hk4vr\" (UID: \"13d4cc11-ed3f-4890-ada9-0f1100dd2de3\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:15.860421 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.860391 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d4cc11-ed3f-4890-ada9-0f1100dd2de3-cert\") pod \"kserve-controller-manager-85d8b8ddfc-hk4vr\" (UID: \"13d4cc11-ed3f-4890-ada9-0f1100dd2de3\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:15.915949 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.915925 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:26:15.961157 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.961095 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d4cc11-ed3f-4890-ada9-0f1100dd2de3-cert\") pod \"kserve-controller-manager-85d8b8ddfc-hk4vr\" (UID: \"13d4cc11-ed3f-4890-ada9-0f1100dd2de3\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:15.961269 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.961167 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkrwd\" (UniqueName: \"kubernetes.io/projected/13d4cc11-ed3f-4890-ada9-0f1100dd2de3-kube-api-access-nkrwd\") pod \"kserve-controller-manager-85d8b8ddfc-hk4vr\" (UID: \"13d4cc11-ed3f-4890-ada9-0f1100dd2de3\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:15.963410 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.963390 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/13d4cc11-ed3f-4890-ada9-0f1100dd2de3-cert\") pod \"kserve-controller-manager-85d8b8ddfc-hk4vr\" (UID: \"13d4cc11-ed3f-4890-ada9-0f1100dd2de3\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:15.970198 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:15.970174 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkrwd\" (UniqueName: \"kubernetes.io/projected/13d4cc11-ed3f-4890-ada9-0f1100dd2de3-kube-api-access-nkrwd\") pod \"kserve-controller-manager-85d8b8ddfc-hk4vr\" (UID: \"13d4cc11-ed3f-4890-ada9-0f1100dd2de3\") " pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:16.055441 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.055415 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:16.062507 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.062485 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g2n7\" (UniqueName: \"kubernetes.io/projected/186b27a3-b7f1-4718-8f5b-938407afdd93-kube-api-access-7g2n7\") pod \"186b27a3-b7f1-4718-8f5b-938407afdd93\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " Jul 28 16:26:16.062616 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.062546 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186b27a3-b7f1-4718-8f5b-938407afdd93-cert\") pod \"186b27a3-b7f1-4718-8f5b-938407afdd93\" (UID: \"186b27a3-b7f1-4718-8f5b-938407afdd93\") " Jul 28 16:26:16.064577 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.064552 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/186b27a3-b7f1-4718-8f5b-938407afdd93-kube-api-access-7g2n7" (OuterVolumeSpecName: "kube-api-access-7g2n7") pod "186b27a3-b7f1-4718-8f5b-938407afdd93" (UID: "186b27a3-b7f1-4718-8f5b-938407afdd93"). InnerVolumeSpecName "kube-api-access-7g2n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:26:16.064633 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.064571 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/186b27a3-b7f1-4718-8f5b-938407afdd93-cert" (OuterVolumeSpecName: "cert") pod "186b27a3-b7f1-4718-8f5b-938407afdd93" (UID: "186b27a3-b7f1-4718-8f5b-938407afdd93"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:26:16.163122 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.163095 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g2n7\" (UniqueName: \"kubernetes.io/projected/186b27a3-b7f1-4718-8f5b-938407afdd93-kube-api-access-7g2n7\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:26:16.163122 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.163119 2574 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/186b27a3-b7f1-4718-8f5b-938407afdd93-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:26:16.179847 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.179771 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-hk4vr"] Jul 28 16:26:16.181947 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:26:16.181921 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d4cc11_ed3f_4890_ada9_0f1100dd2de3.slice/crio-75613bc26068fa48b245a5b0fd4f17488abed30451b9debeee68d28b2f0f08ca WatchSource:0}: Error finding container 75613bc26068fa48b245a5b0fd4f17488abed30451b9debeee68d28b2f0f08ca: Status 404 returned error can't find the container with id 75613bc26068fa48b245a5b0fd4f17488abed30451b9debeee68d28b2f0f08ca Jul 28 16:26:16.337322 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.337295 2574 generic.go:356] "Generic (PLEG): container finished" podID="186b27a3-b7f1-4718-8f5b-938407afdd93" containerID="f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94" exitCode=0 Jul 28 16:26:16.337480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.337344 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" Jul 28 16:26:16.337480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.337383 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" event={"ID":"186b27a3-b7f1-4718-8f5b-938407afdd93","Type":"ContainerDied","Data":"f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94"} Jul 28 16:26:16.337480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.337442 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-6vs58" event={"ID":"186b27a3-b7f1-4718-8f5b-938407afdd93","Type":"ContainerDied","Data":"22ecd73eb7e50c40774d07fe4fc190a4a25da90059c484e9a5dfcfaba95c97e8"} Jul 28 16:26:16.337480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.337472 2574 scope.go:117] "RemoveContainer" containerID="f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94" Jul 28 16:26:16.338617 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.338548 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" event={"ID":"13d4cc11-ed3f-4890-ada9-0f1100dd2de3","Type":"ContainerStarted","Data":"75613bc26068fa48b245a5b0fd4f17488abed30451b9debeee68d28b2f0f08ca"} Jul 28 16:26:16.346706 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.346685 2574 scope.go:117] "RemoveContainer" containerID="f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94" Jul 28 16:26:16.347013 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:26:16.346995 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94\": container with ID starting with f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94 not found: ID does not exist" containerID="f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94" Jul 28 16:26:16.347072 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.347020 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94"} err="failed to get container status \"f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94\": rpc error: code = NotFound desc = could not find container \"f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94\": container with ID starting with f60c13bcbc230e37c302b7beef2605322b0abc0d0074f8b1297595763f5d3f94 not found: ID does not exist" Jul 28 16:26:16.361157 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.361129 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-6vs58"] Jul 28 16:26:16.366685 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:16.366660 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-85d8b8ddfc-6vs58"] Jul 28 16:26:17.344873 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:17.344833 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" event={"ID":"13d4cc11-ed3f-4890-ada9-0f1100dd2de3","Type":"ContainerStarted","Data":"25ee9ede883da8491f571c77318e7147a095a6e0df77e73379ad6fc66239c2f6"} Jul 28 16:26:17.345340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:17.344995 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:17.364742 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:17.364680 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" podStartSLOduration=1.900968256 podStartE2EDuration="2.364666091s" totalImagesPullingTime="463.697835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:26:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:26:17.362953156 +0000 UTC m=+382.221492922" watchObservedRunningTime="2026-07-28 16:26:17.364666091 +0000 UTC m=+382.223205833" Jul 28 16:26:17.700692 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:17.700625 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="186b27a3-b7f1-4718-8f5b-938407afdd93" path="/var/lib/kubelet/pods/186b27a3-b7f1-4718-8f5b-938407afdd93/volumes" Jul 28 16:26:26.679393 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:26.679358 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:26.780451 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:26.780424 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:29.988525 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:29.988487 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69fc7f8776-vb5n2"] Jul 28 16:26:29.988901 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:29.988865 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="186b27a3-b7f1-4718-8f5b-938407afdd93" containerName="manager" Jul 28 16:26:29.988901 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:29.988878 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="186b27a3-b7f1-4718-8f5b-938407afdd93" containerName="manager" Jul 28 16:26:29.988983 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:29.988971 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="186b27a3-b7f1-4718-8f5b-938407afdd93" containerName="manager" Jul 28 16:26:29.990995 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:29.990977 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.002841 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.002818 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69fc7f8776-vb5n2"] Jul 28 16:26:30.057349 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.057314 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:30.072620 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.072593 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-trusted-ca-bundle\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.072767 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.072623 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87kgf\" (UniqueName: \"kubernetes.io/projected/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-kube-api-access-87kgf\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.072767 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.072650 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-config\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.072892 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.072767 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-service-ca\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.072892 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.072836 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-oauth-config\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.072892 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.072864 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-oauth-serving-cert\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.073013 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.072896 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-serving-cert\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.156358 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.156325 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:30.173558 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.173535 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-trusted-ca-bundle\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.173627 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.173565 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87kgf\" (UniqueName: \"kubernetes.io/projected/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-kube-api-access-87kgf\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.173627 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.173585 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-config\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.173627 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.173612 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-service-ca\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.173871 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.173664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-oauth-config\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.173871 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.173686 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-oauth-serving-cert\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.173871 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.173735 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-serving-cert\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.174429 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.174400 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-service-ca\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.174553 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.174531 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-config\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.174630 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.174546 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-trusted-ca-bundle\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.174630 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.174546 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-oauth-serving-cert\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.176192 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.176168 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-serving-cert\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.176285 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.176225 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-console-oauth-config\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.183103 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.183084 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87kgf\" (UniqueName: \"kubernetes.io/projected/269faaf4-cbc6-4035-841f-1ce75f8f5b1c-kube-api-access-87kgf\") pod \"console-69fc7f8776-vb5n2\" (UID: \"269faaf4-cbc6-4035-841f-1ce75f8f5b1c\") " pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.299727 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.299650 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:30.432600 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:30.432572 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69fc7f8776-vb5n2"] Jul 28 16:26:30.434663 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:26:30.434639 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269faaf4_cbc6_4035_841f_1ce75f8f5b1c.slice/crio-aa258900eeab69b05ec2e81e5846bf30f5fa2a63bfbdf3c61c32e18b880b3c88 WatchSource:0}: Error finding container aa258900eeab69b05ec2e81e5846bf30f5fa2a63bfbdf3c61c32e18b880b3c88: Status 404 returned error can't find the container with id aa258900eeab69b05ec2e81e5846bf30f5fa2a63bfbdf3c61c32e18b880b3c88 Jul 28 16:26:31.406428 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:31.406396 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fc7f8776-vb5n2" event={"ID":"269faaf4-cbc6-4035-841f-1ce75f8f5b1c","Type":"ContainerStarted","Data":"3c0378c5308b4dff65222e957e79288c83726bb253427c3c8b3a96f818c1b362"} Jul 28 16:26:31.406838 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:31.406445 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69fc7f8776-vb5n2" event={"ID":"269faaf4-cbc6-4035-841f-1ce75f8f5b1c","Type":"ContainerStarted","Data":"aa258900eeab69b05ec2e81e5846bf30f5fa2a63bfbdf3c61c32e18b880b3c88"} Jul 28 16:26:31.428583 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:31.428536 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69fc7f8776-vb5n2" podStartSLOduration=2.428521559 podStartE2EDuration="2.428521559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:26:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:26:31.426751763 +0000 UTC m=+396.285291506" watchObservedRunningTime="2026-07-28 16:26:31.428521559 +0000 UTC m=+396.287061302" Jul 28 16:26:40.300436 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.300399 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:40.301009 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.300478 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:40.305205 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.305181 2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:40.447222 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.447194 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69fc7f8776-vb5n2" Jul 28 16:26:40.506322 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.506292 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65744cc56b-pfdn8"] Jul 28 16:26:40.642897 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.642817 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:40.744401 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.744371 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:40.844411 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:40.844385 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:48.353305 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:48.353272 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-85d8b8ddfc-hk4vr" Jul 28 16:26:49.393175 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:49.393139 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:49.478248 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:49.478217 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:49.734061 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:49.734030 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:50.113799 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:50.113690 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:50.905873 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:50.905838 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:26:51.041779 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:26:51.041740 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:05.530815 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.530763 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-65744cc56b-pfdn8" podUID="476447dd-7140-4ec1-9a93-8092c76d3476" containerName="console" containerID="cri-o://eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2" gracePeriod=15 Jul 28 16:27:05.779663 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.779640 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:27:05.860340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860274 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-oauth-config\") pod \"476447dd-7140-4ec1-9a93-8092c76d3476\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " Jul 28 16:27:05.860340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860308 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-serving-cert\") pod \"476447dd-7140-4ec1-9a93-8092c76d3476\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " Jul 28 16:27:05.860340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860337 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-trusted-ca-bundle\") pod \"476447dd-7140-4ec1-9a93-8092c76d3476\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " Jul 28 16:27:05.860556 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860370 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-service-ca\") pod \"476447dd-7140-4ec1-9a93-8092c76d3476\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " Jul 28 16:27:05.860556 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860396 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-console-config\") pod \"476447dd-7140-4ec1-9a93-8092c76d3476\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " Jul 28 16:27:05.860556 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860466 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4l57\" (UniqueName: \"kubernetes.io/projected/476447dd-7140-4ec1-9a93-8092c76d3476-kube-api-access-d4l57\") pod \"476447dd-7140-4ec1-9a93-8092c76d3476\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " Jul 28 16:27:05.860556 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860525 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-oauth-serving-cert\") pod \"476447dd-7140-4ec1-9a93-8092c76d3476\" (UID: \"476447dd-7140-4ec1-9a93-8092c76d3476\") " Jul 28 16:27:05.860817 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860778 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-service-ca" (OuterVolumeSpecName: "service-ca") pod "476447dd-7140-4ec1-9a93-8092c76d3476" (UID: "476447dd-7140-4ec1-9a93-8092c76d3476"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:27:05.860817 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860791 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "476447dd-7140-4ec1-9a93-8092c76d3476" (UID: "476447dd-7140-4ec1-9a93-8092c76d3476"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:27:05.860961 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860870 2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-trusted-ca-bundle\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:27:05.860961 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.860887 2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-service-ca\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:27:05.861063 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.861022 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "476447dd-7140-4ec1-9a93-8092c76d3476" (UID: "476447dd-7140-4ec1-9a93-8092c76d3476"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:27:05.861173 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.861149 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-console-config" (OuterVolumeSpecName: "console-config") pod "476447dd-7140-4ec1-9a93-8092c76d3476" (UID: "476447dd-7140-4ec1-9a93-8092c76d3476"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:27:05.862480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.862455 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "476447dd-7140-4ec1-9a93-8092c76d3476" (UID: "476447dd-7140-4ec1-9a93-8092c76d3476"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:27:05.862598 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.862522 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "476447dd-7140-4ec1-9a93-8092c76d3476" (UID: "476447dd-7140-4ec1-9a93-8092c76d3476"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:27:05.862598 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.862539 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/476447dd-7140-4ec1-9a93-8092c76d3476-kube-api-access-d4l57" (OuterVolumeSpecName: "kube-api-access-d4l57") pod "476447dd-7140-4ec1-9a93-8092c76d3476" (UID: "476447dd-7140-4ec1-9a93-8092c76d3476"). InnerVolumeSpecName "kube-api-access-d4l57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:27:05.961854 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.961834 2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-console-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:27:05.961972 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.961854 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4l57\" (UniqueName: \"kubernetes.io/projected/476447dd-7140-4ec1-9a93-8092c76d3476-kube-api-access-d4l57\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:27:05.961972 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.961872 2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/476447dd-7140-4ec1-9a93-8092c76d3476-oauth-serving-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:27:05.961972 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.961881 2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-oauth-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:27:05.961972 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:05.961890 2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/476447dd-7140-4ec1-9a93-8092c76d3476-console-serving-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:27:06.553577 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.553544 2574 generic.go:356] "Generic (PLEG): container finished" podID="476447dd-7140-4ec1-9a93-8092c76d3476" containerID="eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2" exitCode=0 Jul 28 16:27:06.554018 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.553588 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65744cc56b-pfdn8" Jul 28 16:27:06.554018 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.553613 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65744cc56b-pfdn8" event={"ID":"476447dd-7140-4ec1-9a93-8092c76d3476","Type":"ContainerDied","Data":"eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2"} Jul 28 16:27:06.554018 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.553652 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65744cc56b-pfdn8" event={"ID":"476447dd-7140-4ec1-9a93-8092c76d3476","Type":"ContainerDied","Data":"98d6685c52fdfc227d1571bcc3d165ac47a40f611dc1f60fb5cdb5fb915e16f2"} Jul 28 16:27:06.554018 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.553671 2574 scope.go:117] "RemoveContainer" containerID="eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2" Jul 28 16:27:06.562193 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.562175 2574 scope.go:117] "RemoveContainer" containerID="eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2" Jul 28 16:27:06.562446 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:27:06.562427 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2\": container with ID starting with eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2 not found: ID does not exist" containerID="eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2" Jul 28 16:27:06.562492 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.562454 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2"} err="failed to get container status \"eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2\": rpc error: code = NotFound desc = could not find container \"eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2\": container with ID starting with eb49675f7d3d2c9aa6cc0b34513f4bf8ffccb2512b7723f1d24ad22d88df7dd2 not found: ID does not exist" Jul 28 16:27:06.575503 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.575482 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65744cc56b-pfdn8"] Jul 28 16:27:06.581244 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:06.581221 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65744cc56b-pfdn8"] Jul 28 16:27:07.700472 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:07.700439 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="476447dd-7140-4ec1-9a93-8092c76d3476" path="/var/lib/kubelet/pods/476447dd-7140-4ec1-9a93-8092c76d3476/volumes" Jul 28 16:27:17.192627 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:17.192550 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:31.753846 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:31.753812 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:27:47.778228 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:27:47.778189 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:28:24.441974 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:28:24.441940 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:28:24.544095 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:28:24.544061 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:28:36.014011 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:28:36.013979 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:28:36.368494 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:28:36.368401 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:00.600231 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:00.600192 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:02.338152 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:02.338115 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:11.316818 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:11.316766 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:11.416583 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:11.416551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:11.497147 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:11.497104 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:11.579943 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:11.579863 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:11.810519 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:11.810484 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:30:55.324561 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:30:55.324524 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:41.381485 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.381449 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj"] Jul 28 16:31:41.381999 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.381973 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476447dd-7140-4ec1-9a93-8092c76d3476" containerName="console" Jul 28 16:31:41.381999 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.381992 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="476447dd-7140-4ec1-9a93-8092c76d3476" containerName="console" Jul 28 16:31:41.382112 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.382075 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="476447dd-7140-4ec1-9a93-8092c76d3476" containerName="console" Jul 28 16:31:41.383925 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.383905 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.388191 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.388168 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-3b17e-serving-cert\"" Jul 28 16:31:41.388322 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.388176 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jlwpj\"" Jul 28 16:31:41.388671 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.388650 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-3b17e-kube-rbac-proxy-sar-config\"" Jul 28 16:31:41.389246 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.389231 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 16:31:41.394978 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.394958 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj"] Jul 28 16:31:41.402235 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.402208 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:41.485820 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.485786 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d22d3de-b8eb-4508-b870-f548779e1877-openshift-service-ca-bundle\") pod \"model-chainer-raw-3b17e-776bc4b57f-v47hj\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.485983 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.485837 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d22d3de-b8eb-4508-b870-f548779e1877-proxy-tls\") pod \"model-chainer-raw-3b17e-776bc4b57f-v47hj\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.499086 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.499058 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:41.586649 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.586615 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d22d3de-b8eb-4508-b870-f548779e1877-openshift-service-ca-bundle\") pod \"model-chainer-raw-3b17e-776bc4b57f-v47hj\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.586811 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.586664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d22d3de-b8eb-4508-b870-f548779e1877-proxy-tls\") pod \"model-chainer-raw-3b17e-776bc4b57f-v47hj\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.587330 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.587307 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d22d3de-b8eb-4508-b870-f548779e1877-openshift-service-ca-bundle\") pod \"model-chainer-raw-3b17e-776bc4b57f-v47hj\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.588988 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.588965 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d22d3de-b8eb-4508-b870-f548779e1877-proxy-tls\") pod \"model-chainer-raw-3b17e-776bc4b57f-v47hj\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.694259 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.694167 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:41.819935 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.819903 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj"] Jul 28 16:31:41.823304 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:31:41.823278 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d22d3de_b8eb_4508_b870_f548779e1877.slice/crio-3e054a4260d73ed7531c2b456a75a99bd75c1ff963ed2fef09b0cf7acd312c5e WatchSource:0}: Error finding container 3e054a4260d73ed7531c2b456a75a99bd75c1ff963ed2fef09b0cf7acd312c5e: Status 404 returned error can't find the container with id 3e054a4260d73ed7531c2b456a75a99bd75c1ff963ed2fef09b0cf7acd312c5e Jul 28 16:31:41.825638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:41.825620 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:31:42.718290 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:42.718242 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" event={"ID":"8d22d3de-b8eb-4508-b870-f548779e1877","Type":"ContainerStarted","Data":"3e054a4260d73ed7531c2b456a75a99bd75c1ff963ed2fef09b0cf7acd312c5e"} Jul 28 16:31:44.730396 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:44.730336 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" event={"ID":"8d22d3de-b8eb-4508-b870-f548779e1877","Type":"ContainerStarted","Data":"c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749"} Jul 28 16:31:44.730879 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:44.730417 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:44.747442 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:44.747391 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podStartSLOduration=1.24661377 podStartE2EDuration="3.747374984s" totalImagesPullingTime="2.500761214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:31:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:31:44.746698973 +0000 UTC m=+709.605238715" watchObservedRunningTime="2026-07-28 16:31:44.747374984 +0000 UTC m=+709.605914727" Jul 28 16:31:50.740911 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:50.740837 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:31:51.432514 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:51.432479 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj"] Jul 28 16:31:51.432687 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:51.432667 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" containerID="cri-o://c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749" gracePeriod=30 Jul 28 16:31:51.454608 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:51.454581 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:51.611245 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:51.611213 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:51.705518 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:51.705444 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:51.796654 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:51.796622 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:51.929047 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:51.929009 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:52.053236 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:52.053208 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:52.156471 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:52.156437 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:31:55.739326 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:31:55.739285 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:32:00.739061 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:00.739018 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:32:05.739644 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:05.739601 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:32:05.740084 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:05.739731 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:32:07.153896 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:07.153856 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:32:07.254428 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:07.254398 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:32:10.739694 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:10.739659 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:32:15.739538 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:15.739493 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:32:20.739480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:20.739435 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:32:21.583329 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.583304 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:32:21.616330 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.616297 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d22d3de-b8eb-4508-b870-f548779e1877-openshift-service-ca-bundle\") pod \"8d22d3de-b8eb-4508-b870-f548779e1877\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " Jul 28 16:32:21.616475 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.616418 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d22d3de-b8eb-4508-b870-f548779e1877-proxy-tls\") pod \"8d22d3de-b8eb-4508-b870-f548779e1877\" (UID: \"8d22d3de-b8eb-4508-b870-f548779e1877\") " Jul 28 16:32:21.616780 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.616749 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d22d3de-b8eb-4508-b870-f548779e1877-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "8d22d3de-b8eb-4508-b870-f548779e1877" (UID: "8d22d3de-b8eb-4508-b870-f548779e1877"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:32:21.618545 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.618519 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d22d3de-b8eb-4508-b870-f548779e1877-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "8d22d3de-b8eb-4508-b870-f548779e1877" (UID: "8d22d3de-b8eb-4508-b870-f548779e1877"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:32:21.717530 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.717501 2574 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/8d22d3de-b8eb-4508-b870-f548779e1877-proxy-tls\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:32:21.717530 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.717526 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d22d3de-b8eb-4508-b870-f548779e1877-openshift-service-ca-bundle\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:32:21.894415 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.894374 2574 generic.go:356] "Generic (PLEG): container finished" podID="8d22d3de-b8eb-4508-b870-f548779e1877" containerID="c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749" exitCode=0 Jul 28 16:32:21.894848 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.894434 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" Jul 28 16:32:21.894848 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.894461 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" event={"ID":"8d22d3de-b8eb-4508-b870-f548779e1877","Type":"ContainerDied","Data":"c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749"} Jul 28 16:32:21.894848 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.894522 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj" event={"ID":"8d22d3de-b8eb-4508-b870-f548779e1877","Type":"ContainerDied","Data":"3e054a4260d73ed7531c2b456a75a99bd75c1ff963ed2fef09b0cf7acd312c5e"} Jul 28 16:32:21.894848 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.894544 2574 scope.go:117] "RemoveContainer" containerID="c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749" Jul 28 16:32:21.902816 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.902794 2574 scope.go:117] "RemoveContainer" containerID="c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749" Jul 28 16:32:21.903101 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:32:21.903078 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749\": container with ID starting with c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749 not found: ID does not exist" containerID="c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749" Jul 28 16:32:21.903167 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.903109 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749"} err="failed to get container status \"c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749\": rpc error: code = NotFound desc = could not find container \"c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749\": container with ID starting with c3f3dcfde5b671b75edac16ec360bb47ded941523faac13a314157795944e749 not found: ID does not exist" Jul 28 16:32:21.912224 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.912199 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj"] Jul 28 16:32:21.915946 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:21.915922 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-3b17e-776bc4b57f-v47hj"] Jul 28 16:32:23.700978 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:32:23.700946 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" path="/var/lib/kubelet/pods/8d22d3de-b8eb-4508-b870-f548779e1877/volumes" Jul 28 16:33:08.117551 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:08.117498 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:31.718281 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.718247 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns"] Jul 28 16:33:31.718753 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.718588 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" Jul 28 16:33:31.718753 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.718600 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" Jul 28 16:33:31.718753 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.718662 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d22d3de-b8eb-4508-b870-f548779e1877" containerName="model-chainer-raw-3b17e" Jul 28 16:33:31.720734 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.720700 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:31.721866 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.721826 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:31.723705 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.723689 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 16:33:31.723793 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.723735 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-0983c-serving-cert\"" Jul 28 16:33:31.724662 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.724641 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-0983c-kube-rbac-proxy-sar-config\"" Jul 28 16:33:31.724662 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.724658 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jlwpj\"" Jul 28 16:33:31.733499 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.733474 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns"] Jul 28 16:33:31.801901 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.801880 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls\") pod \"model-chainer-raw-hpa-0983c-5dd8d85684-xmsns\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:31.802020 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.801915 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-0983c-5dd8d85684-xmsns\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:31.902277 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.902251 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls\") pod \"model-chainer-raw-hpa-0983c-5dd8d85684-xmsns\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:31.902384 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.902295 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-0983c-5dd8d85684-xmsns\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:31.902447 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:33:31.902383 2574 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-serving-cert: secret "model-chainer-raw-hpa-0983c-serving-cert" not found Jul 28 16:33:31.902447 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:33:31.902445 2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls podName:9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7 nodeName:}" failed. No retries permitted until 2026-07-28 16:33:32.402430111 +0000 UTC m=+817.260969835 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls") pod "model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" (UID: "9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7") : secret "model-chainer-raw-hpa-0983c-serving-cert" not found Jul 28 16:33:31.902951 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:31.902930 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-0983c-5dd8d85684-xmsns\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:32.140778 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:32.140692 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:32.406260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:32.406206 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls\") pod \"model-chainer-raw-hpa-0983c-5dd8d85684-xmsns\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:32.408554 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:32.408524 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls\") pod \"model-chainer-raw-hpa-0983c-5dd8d85684-xmsns\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:32.630908 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:32.630870 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:32.754757 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:32.754734 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns"] Jul 28 16:33:32.756823 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:33:32.756790 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ce1b21e_8d55_41f1_bb0e_2c37bd872ec7.slice/crio-ffe5c0f117909903b8060df64aa1961b712c438dd527f2591f64c918912178c7 WatchSource:0}: Error finding container ffe5c0f117909903b8060df64aa1961b712c438dd527f2591f64c918912178c7: Status 404 returned error can't find the container with id ffe5c0f117909903b8060df64aa1961b712c438dd527f2591f64c918912178c7 Jul 28 16:33:33.199234 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:33.199201 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" event={"ID":"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7","Type":"ContainerStarted","Data":"2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e"} Jul 28 16:33:33.199404 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:33.199260 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" event={"ID":"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7","Type":"ContainerStarted","Data":"ffe5c0f117909903b8060df64aa1961b712c438dd527f2591f64c918912178c7"} Jul 28 16:33:33.199404 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:33.199306 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:33.216554 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:33.216503 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podStartSLOduration=2.216486337 podStartE2EDuration="2.216486337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:33:33.215802399 +0000 UTC m=+818.074342133" watchObservedRunningTime="2026-07-28 16:33:33.216486337 +0000 UTC m=+818.075026080" Jul 28 16:33:39.208833 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:39.208799 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:41.759237 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:41.759205 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns"] Jul 28 16:33:41.759621 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:41.759487 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" containerID="cri-o://2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e" gracePeriod=30 Jul 28 16:33:41.782241 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:41.782210 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:41.948703 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:41.948677 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:42.028142 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:42.028077 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:42.306906 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:42.306828 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:42.594615 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:42.594544 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:44.207076 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:44.207036 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:33:49.206596 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:49.206550 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:33:51.998931 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:51.998900 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:52.357970 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:52.357884 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:33:54.207741 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:54.207689 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:33:54.208114 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:54.207825 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:33:59.207653 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:33:59.207617 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:34:04.207566 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:04.207525 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:34:09.206968 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:09.206931 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 16:34:11.896635 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:11.896606 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:34:11.997905 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:11.997878 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls\") pod \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " Jul 28 16:34:11.998042 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:11.997919 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-openshift-service-ca-bundle\") pod \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\" (UID: \"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7\") " Jul 28 16:34:11.998260 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:11.998240 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" (UID: "9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:34:11.999861 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:11.999838 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" (UID: "9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:34:12.099123 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.099051 2574 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-proxy-tls\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:34:12.099123 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.099079 2574 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7-openshift-service-ca-bundle\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:34:12.373103 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.373011 2574 generic.go:356] "Generic (PLEG): container finished" podID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerID="2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e" exitCode=0 Jul 28 16:34:12.373103 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.373056 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" event={"ID":"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7","Type":"ContainerDied","Data":"2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e"} Jul 28 16:34:12.373103 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.373069 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" Jul 28 16:34:12.373376 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.373117 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns" event={"ID":"9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7","Type":"ContainerDied","Data":"ffe5c0f117909903b8060df64aa1961b712c438dd527f2591f64c918912178c7"} Jul 28 16:34:12.373376 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.373138 2574 scope.go:117] "RemoveContainer" containerID="2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e" Jul 28 16:34:12.381477 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.381461 2574 scope.go:117] "RemoveContainer" containerID="2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e" Jul 28 16:34:12.381788 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:34:12.381767 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e\": container with ID starting with 2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e not found: ID does not exist" containerID="2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e" Jul 28 16:34:12.381859 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.381796 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e"} err="failed to get container status \"2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e\": rpc error: code = NotFound desc = could not find container \"2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e\": container with ID starting with 2a4b70277cdb80d7251d69cd82e7b95d883577e5643bcd7faec3ded68d24fc7e not found: ID does not exist" Jul 28 16:34:12.395529 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.395508 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns"] Jul 28 16:34:12.399248 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:12.399229 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-0983c-5dd8d85684-xmsns"] Jul 28 16:34:13.700554 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:13.700520 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" path="/var/lib/kubelet/pods/9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7/volumes" Jul 28 16:34:55.283480 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:34:55.283391 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:35:27.309265 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:35:27.309230 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:35:29.450792 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:35:29.450753 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:37:57.634518 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:37:57.634433 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:38:02.675450 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:38:02.675411 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:38:02.860597 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:38:02.860569 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:38:02.960357 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:38:02.960294 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:38:03.061038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:38:03.060997 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:38:14.926437 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:38:14.926390 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:38:20.976312 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:38:20.976273 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:39:27.848667 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:39:27.848574 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:39:27.918443 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:39:27.918415 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:39:32.965436 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:39:32.965405 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:39:33.064004 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:39:33.063975 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:39:33.162767 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:39:33.162735 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:39:33.327371 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:39:33.327301 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:41:03.157549 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.157481 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:41:03.203986 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.203950 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h"] Jul 28 16:41:03.204328 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.204312 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" Jul 28 16:41:03.204375 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.204331 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" Jul 28 16:41:03.204415 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.204386 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ce1b21e-8d55-41f1-bb0e-2c37bd872ec7" containerName="model-chainer-raw-hpa-0983c" Jul 28 16:41:03.206265 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.206249 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.208864 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.208844 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-v2-predictor-serving-cert\"" Jul 28 16:41:03.209189 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.209174 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 28 16:41:03.209644 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.209626 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 28 16:41:03.209790 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.209749 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-promote-kube-rbac-proxy-sar-config\"" Jul 28 16:41:03.209925 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.209900 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jlwpj\"" Jul 28 16:41:03.210033 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.209903 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 28 16:41:03.210033 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.209961 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 16:41:03.223282 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.223259 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h"] Jul 28 16:41:03.321834 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.321812 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/50896c6e-d465-4d22-a3ad-21440806e9bf-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.321951 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.321848 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn5nl\" (UniqueName: \"kubernetes.io/projected/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-api-access-cn5nl\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.321951 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.321874 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50896c6e-d465-4d22-a3ad-21440806e9bf-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.322024 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.321955 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.322024 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.321982 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.422661 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.422602 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/50896c6e-d465-4d22-a3ad-21440806e9bf-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.422661 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.422637 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn5nl\" (UniqueName: \"kubernetes.io/projected/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-api-access-cn5nl\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.422825 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.422664 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50896c6e-d465-4d22-a3ad-21440806e9bf-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.422825 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.422695 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.422825 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.422742 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.423099 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.423077 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/50896c6e-d465-4d22-a3ad-21440806e9bf-kserve-provision-location\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.423372 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.423354 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-cabundle-cert\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.423442 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.423398 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-rbac-proxy-sar-config\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.425136 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.425118 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50896c6e-d465-4d22-a3ad-21440806e9bf-proxy-tls\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.432374 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.432355 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn5nl\" (UniqueName: \"kubernetes.io/projected/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-api-access-cn5nl\") pod \"isvc-canary-promote-v2-predictor-86766c4958-g972h\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.516170 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.516145 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:03.648038 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.648006 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h"] Jul 28 16:41:03.650962 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:41:03.650931 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50896c6e_d465_4d22_a3ad_21440806e9bf.slice/crio-f15118eb33b94e4ef4daab888a1627f72c88cbd59c097a4d52923eb1eb381f57 WatchSource:0}: Error finding container f15118eb33b94e4ef4daab888a1627f72c88cbd59c097a4d52923eb1eb381f57: Status 404 returned error can't find the container with id f15118eb33b94e4ef4daab888a1627f72c88cbd59c097a4d52923eb1eb381f57 Jul 28 16:41:03.655677 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:03.655658 2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 16:41:04.204876 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:04.204844 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerStarted","Data":"f15118eb33b94e4ef4daab888a1627f72c88cbd59c097a4d52923eb1eb381f57"} Jul 28 16:41:08.225665 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:08.225623 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerStarted","Data":"8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264"} Jul 28 16:41:11.241040 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:11.241009 2574 generic.go:356] "Generic (PLEG): container finished" podID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerID="8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264" exitCode=0 Jul 28 16:41:11.241345 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:11.241072 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerDied","Data":"8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264"} Jul 28 16:41:24.316102 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:24.316062 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerStarted","Data":"c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff"} Jul 28 16:41:27.331749 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:27.331690 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerStarted","Data":"43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a"} Jul 28 16:41:27.332166 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:27.331925 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:27.332166 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:27.332083 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:27.333011 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:27.332986 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:41:27.354641 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:27.354601 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podStartSLOduration=1.766439095 podStartE2EDuration="24.35458891s" totalImagesPullingTime="18.891155103s" totalInitContainerRuntime="3.696994712s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:41:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:41:27.353330909 +0000 UTC m=+1292.211870652" watchObservedRunningTime="2026-07-28 16:41:27.35458891 +0000 UTC m=+1292.213128697" Jul 28 16:41:28.336004 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:28.335960 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:41:33.341043 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:33.341013 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:41:33.341450 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:33.341421 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:41:43.342194 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:43.342153 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:41:53.341850 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:41:53.341810 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:42:03.341921 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:03.341875 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:42:13.341924 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:13.341881 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:42:23.341636 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:23.341552 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 28 16:42:33.341800 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:33.341769 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:42:43.543604 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:43.543571 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h"] Jul 28 16:42:43.543980 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:43.543836 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" containerID="cri-o://c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff" gracePeriod=30 Jul 28 16:42:43.543980 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:43.543907 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kube-rbac-proxy" containerID="cri-o://43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a" gracePeriod=30 Jul 28 16:42:43.671556 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:43.671533 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:42:44.656320 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:44.656283 2574 generic.go:356] "Generic (PLEG): container finished" podID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerID="43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a" exitCode=2 Jul 28 16:42:44.656674 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:44.656350 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerDied","Data":"43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a"} Jul 28 16:42:47.483720 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.483682 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:42:47.524307 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524277 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-cabundle-cert\") pod \"50896c6e-d465-4d22-a3ad-21440806e9bf\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " Jul 28 16:42:47.524463 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524338 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/50896c6e-d465-4d22-a3ad-21440806e9bf-kserve-provision-location\") pod \"50896c6e-d465-4d22-a3ad-21440806e9bf\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " Jul 28 16:42:47.524463 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524373 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn5nl\" (UniqueName: \"kubernetes.io/projected/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-api-access-cn5nl\") pod \"50896c6e-d465-4d22-a3ad-21440806e9bf\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " Jul 28 16:42:47.524463 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524406 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-rbac-proxy-sar-config\") pod \"50896c6e-d465-4d22-a3ad-21440806e9bf\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " Jul 28 16:42:47.524664 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524510 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50896c6e-d465-4d22-a3ad-21440806e9bf-proxy-tls\") pod \"50896c6e-d465-4d22-a3ad-21440806e9bf\" (UID: \"50896c6e-d465-4d22-a3ad-21440806e9bf\") " Jul 28 16:42:47.524797 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524764 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50896c6e-d465-4d22-a3ad-21440806e9bf-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "50896c6e-d465-4d22-a3ad-21440806e9bf" (UID: "50896c6e-d465-4d22-a3ad-21440806e9bf"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:42:47.524913 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524868 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "50896c6e-d465-4d22-a3ad-21440806e9bf" (UID: "50896c6e-d465-4d22-a3ad-21440806e9bf"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:42:47.525015 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.524960 2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-cabundle-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:42:47.525015 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.525004 2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/50896c6e-d465-4d22-a3ad-21440806e9bf-kserve-provision-location\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:42:47.525160 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.525118 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "50896c6e-d465-4d22-a3ad-21440806e9bf" (UID: "50896c6e-d465-4d22-a3ad-21440806e9bf"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:42:47.527213 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.527181 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50896c6e-d465-4d22-a3ad-21440806e9bf-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "50896c6e-d465-4d22-a3ad-21440806e9bf" (UID: "50896c6e-d465-4d22-a3ad-21440806e9bf"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:42:47.527324 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.527259 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-api-access-cn5nl" (OuterVolumeSpecName: "kube-api-access-cn5nl") pod "50896c6e-d465-4d22-a3ad-21440806e9bf" (UID: "50896c6e-d465-4d22-a3ad-21440806e9bf"). InnerVolumeSpecName "kube-api-access-cn5nl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:42:47.626072 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.626024 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn5nl\" (UniqueName: \"kubernetes.io/projected/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-api-access-cn5nl\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:42:47.626072 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.626045 2574 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/50896c6e-d465-4d22-a3ad-21440806e9bf-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:42:47.626072 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.626055 2574 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/50896c6e-d465-4d22-a3ad-21440806e9bf-proxy-tls\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:42:47.672545 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.672508 2574 generic.go:356] "Generic (PLEG): container finished" podID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerID="c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff" exitCode=0 Jul 28 16:42:47.672637 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.672549 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerDied","Data":"c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff"} Jul 28 16:42:47.672637 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.672565 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" Jul 28 16:42:47.672637 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.672610 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h" event={"ID":"50896c6e-d465-4d22-a3ad-21440806e9bf","Type":"ContainerDied","Data":"f15118eb33b94e4ef4daab888a1627f72c88cbd59c097a4d52923eb1eb381f57"} Jul 28 16:42:47.672637 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.672637 2574 scope.go:117] "RemoveContainer" containerID="43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a" Jul 28 16:42:47.681228 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.681214 2574 scope.go:117] "RemoveContainer" containerID="c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff" Jul 28 16:42:47.688300 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.688281 2574 scope.go:117] "RemoveContainer" containerID="8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264" Jul 28 16:42:47.695055 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.695036 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h"] Jul 28 16:42:47.695421 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.695406 2574 scope.go:117] "RemoveContainer" containerID="43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a" Jul 28 16:42:47.695689 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:42:47.695664 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a\": container with ID starting with 43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a not found: ID does not exist" containerID="43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a" Jul 28 16:42:47.695794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.695695 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a"} err="failed to get container status \"43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a\": rpc error: code = NotFound desc = could not find container \"43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a\": container with ID starting with 43ca86193318742b9f4bcb1f71843c32ace262b7aa06a96b54e44abe9e3fca3a not found: ID does not exist" Jul 28 16:42:47.695794 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.695728 2574 scope.go:117] "RemoveContainer" containerID="c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff" Jul 28 16:42:47.695968 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:42:47.695951 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff\": container with ID starting with c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff not found: ID does not exist" containerID="c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff" Jul 28 16:42:47.696004 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.695976 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff"} err="failed to get container status \"c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff\": rpc error: code = NotFound desc = could not find container \"c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff\": container with ID starting with c7b650a823508b01f0813971d630d584244a589613fc56071aa10d8af71bfbff not found: ID does not exist" Jul 28 16:42:47.696004 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.695991 2574 scope.go:117] "RemoveContainer" containerID="8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264" Jul 28 16:42:47.696204 ip-10-0-132-37 kubenswrapper[2574]: E0728 16:42:47.696183 2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264\": container with ID starting with 8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264 not found: ID does not exist" containerID="8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264" Jul 28 16:42:47.696258 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.696213 2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264"} err="failed to get container status \"8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264\": rpc error: code = NotFound desc = could not find container \"8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264\": container with ID starting with 8822159f543c13ef2bff28ce45672a4008cf9da4fa7f84161ddd95c6c389b264 not found: ID does not exist" Jul 28 16:42:47.700857 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:47.700838 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-promote-v2-predictor-86766c4958-g972h"] Jul 28 16:42:48.541257 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:48.541197 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:42:48.621366 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:48.621335 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:42:48.695694 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:48.695663 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:42:48.957942 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:48.957872 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:42:49.700422 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:42:49.700380 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" path="/var/lib/kubelet/pods/50896c6e-d465-4d22-a3ad-21440806e9bf/volumes" Jul 28 16:43:19.703893 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:43:19.703846 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:44:08.628414 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.628341 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:44:08.645991 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.645959 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64"] Jul 28 16:44:08.646296 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646283 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="storage-initializer" Jul 28 16:44:08.646337 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646297 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="storage-initializer" Jul 28 16:44:08.646337 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646305 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" Jul 28 16:44:08.646337 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646311 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" Jul 28 16:44:08.646337 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646322 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kube-rbac-proxy" Jul 28 16:44:08.646337 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646328 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kube-rbac-proxy" Jul 28 16:44:08.646478 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646396 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kserve-container" Jul 28 16:44:08.646478 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.646408 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50896c6e-d465-4d22-a3ad-21440806e9bf" containerName="kube-rbac-proxy" Jul 28 16:44:08.649349 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.649333 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.654347 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.654289 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 28 16:44:08.654496 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.654453 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-predictor-serving-cert\"" Jul 28 16:44:08.658767 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.658745 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 28 16:44:08.658857 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.658759 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 28 16:44:08.658910 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.658886 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-jlwpj\"" Jul 28 16:44:08.667151 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.667135 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 16:44:08.668820 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.668805 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-canary-rollback-kube-rbac-proxy-sar-config\"" Jul 28 16:44:08.684500 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.684479 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64"] Jul 28 16:44:08.713157 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.713130 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbce8518-b479-4cdb-a087-e55f6fe77c90-proxy-tls\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.713249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.713160 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksp7b\" (UniqueName: \"kubernetes.io/projected/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-api-access-ksp7b\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.713249 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.713207 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bbce8518-b479-4cdb-a087-e55f6fe77c90-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.713322 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.713260 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.713322 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.713294 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.813938 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.813914 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bbce8518-b479-4cdb-a087-e55f6fe77c90-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.814023 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.813943 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.814023 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.813964 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.814023 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.814017 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbce8518-b479-4cdb-a087-e55f6fe77c90-proxy-tls\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.814210 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.814032 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksp7b\" (UniqueName: \"kubernetes.io/projected/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-api-access-ksp7b\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.814348 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.814323 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bbce8518-b479-4cdb-a087-e55f6fe77c90-kserve-provision-location\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.814600 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.814574 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-cabundle-cert\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.814685 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.814623 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-rbac-proxy-sar-config\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.816329 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.816308 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbce8518-b479-4cdb-a087-e55f6fe77c90-proxy-tls\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.823526 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.823505 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksp7b\" (UniqueName: \"kubernetes.io/projected/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-api-access-ksp7b\") pod \"isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:08.958870 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:08.958807 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:09.089241 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:09.089216 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64"] Jul 28 16:44:09.091791 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:44:09.091763 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbce8518_b479_4cdb_a087_e55f6fe77c90.slice/crio-4708c25854a6e425dc7a34c1bc1c2a0e4bcca2ddb7c0dff9cc562da15d507e90 WatchSource:0}: Error finding container 4708c25854a6e425dc7a34c1bc1c2a0e4bcca2ddb7c0dff9cc562da15d507e90: Status 404 returned error can't find the container with id 4708c25854a6e425dc7a34c1bc1c2a0e4bcca2ddb7c0dff9cc562da15d507e90 Jul 28 16:44:10.018833 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:10.018798 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerStarted","Data":"d64edcdb1e38eb8b97f3a82073f6a02ae0b99e8fe7eefd5cc32f3c75e66d40ed"} Jul 28 16:44:10.019176 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:10.018858 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerStarted","Data":"4708c25854a6e425dc7a34c1bc1c2a0e4bcca2ddb7c0dff9cc562da15d507e90"} Jul 28 16:44:13.032169 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:13.032137 2574 generic.go:356] "Generic (PLEG): container finished" podID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerID="d64edcdb1e38eb8b97f3a82073f6a02ae0b99e8fe7eefd5cc32f3c75e66d40ed" exitCode=0 Jul 28 16:44:13.032508 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:13.032209 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerDied","Data":"d64edcdb1e38eb8b97f3a82073f6a02ae0b99e8fe7eefd5cc32f3c75e66d40ed"} Jul 28 16:44:14.038557 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:14.038526 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerStarted","Data":"005f37fe1f7843529248b19314a000f559e045278d382b47ad4a87fd1ee56cea"} Jul 28 16:44:14.038925 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:14.038580 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerStarted","Data":"c7d6ed59e5f126ad338e02758b1d58ab3246307958f2ad146e2b1619f1c2ca57"} Jul 28 16:44:14.038925 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:14.038765 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:14.061804 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:14.061764 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podStartSLOduration=2.521109311 podStartE2EDuration="6.061751718s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.540642407s" isStatefulPod=false podCreationTimestamp="2026-07-28 16:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:44:14.060332773 +0000 UTC m=+1458.918872512" watchObservedRunningTime="2026-07-28 16:44:14.061751718 +0000 UTC m=+1458.920291459" Jul 28 16:44:15.043289 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:15.043257 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:15.044487 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:15.044461 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:44:16.046819 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:16.046782 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:44:21.051097 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:21.051069 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:44:21.051530 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:21.051505 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:44:31.051824 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:31.051784 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:44:41.051494 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:41.051454 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:44:51.051938 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:44:51.051902 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:45:01.051598 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:01.051561 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:45:11.052097 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:11.052061 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:45:21.052616 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:21.052545 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:45:28.760641 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:28.760610 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64"] Jul 28 16:45:28.761106 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:28.760853 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" containerID="cri-o://c7d6ed59e5f126ad338e02758b1d58ab3246307958f2ad146e2b1619f1c2ca57" gracePeriod=30 Jul 28 16:45:28.761106 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:28.760874 2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kube-rbac-proxy" containerID="cri-o://005f37fe1f7843529248b19314a000f559e045278d382b47ad4a87fd1ee56cea" gracePeriod=30 Jul 28 16:45:29.347400 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:29.347368 2574 generic.go:356] "Generic (PLEG): container finished" podID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerID="005f37fe1f7843529248b19314a000f559e045278d382b47ad4a87fd1ee56cea" exitCode=2 Jul 28 16:45:29.347549 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:29.347408 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerDied","Data":"005f37fe1f7843529248b19314a000f559e045278d382b47ad4a87fd1ee56cea"} Jul 28 16:45:29.820515 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:29.820488 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:45:31.047089 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:31.047054 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": dial tcp 10.133.0.32:8643: connect: connection refused" Jul 28 16:45:31.052459 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:31.052433 2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 28 16:45:32.361074 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.361041 2574 generic.go:356] "Generic (PLEG): container finished" podID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerID="c7d6ed59e5f126ad338e02758b1d58ab3246307958f2ad146e2b1619f1c2ca57" exitCode=0 Jul 28 16:45:32.361439 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.361076 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerDied","Data":"c7d6ed59e5f126ad338e02758b1d58ab3246307958f2ad146e2b1619f1c2ca57"} Jul 28 16:45:32.407945 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.407924 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:45:32.437243 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437194 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bbce8518-b479-4cdb-a087-e55f6fe77c90-kserve-provision-location\") pod \"bbce8518-b479-4cdb-a087-e55f6fe77c90\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " Jul 28 16:45:32.437340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437260 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ksp7b\" (UniqueName: \"kubernetes.io/projected/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-api-access-ksp7b\") pod \"bbce8518-b479-4cdb-a087-e55f6fe77c90\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " Jul 28 16:45:32.437340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437299 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-rbac-proxy-sar-config\") pod \"bbce8518-b479-4cdb-a087-e55f6fe77c90\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " Jul 28 16:45:32.437340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437332 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-cabundle-cert\") pod \"bbce8518-b479-4cdb-a087-e55f6fe77c90\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " Jul 28 16:45:32.437494 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437357 2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbce8518-b479-4cdb-a087-e55f6fe77c90-proxy-tls\") pod \"bbce8518-b479-4cdb-a087-e55f6fe77c90\" (UID: \"bbce8518-b479-4cdb-a087-e55f6fe77c90\") " Jul 28 16:45:32.437549 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437491 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbce8518-b479-4cdb-a087-e55f6fe77c90-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "bbce8518-b479-4cdb-a087-e55f6fe77c90" (UID: "bbce8518-b479-4cdb-a087-e55f6fe77c90"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 16:45:32.437635 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437615 2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bbce8518-b479-4cdb-a087-e55f6fe77c90-kserve-provision-location\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:45:32.437813 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437787 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "bbce8518-b479-4cdb-a087-e55f6fe77c90" (UID: "bbce8518-b479-4cdb-a087-e55f6fe77c90"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:45:32.437879 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.437808 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bbce8518-b479-4cdb-a087-e55f6fe77c90" (UID: "bbce8518-b479-4cdb-a087-e55f6fe77c90"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 16:45:32.439346 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.439320 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-api-access-ksp7b" (OuterVolumeSpecName: "kube-api-access-ksp7b") pod "bbce8518-b479-4cdb-a087-e55f6fe77c90" (UID: "bbce8518-b479-4cdb-a087-e55f6fe77c90"). InnerVolumeSpecName "kube-api-access-ksp7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 16:45:32.439433 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.439408 2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbce8518-b479-4cdb-a087-e55f6fe77c90-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bbce8518-b479-4cdb-a087-e55f6fe77c90" (UID: "bbce8518-b479-4cdb-a087-e55f6fe77c90"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 16:45:32.538340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.538306 2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksp7b\" (UniqueName: \"kubernetes.io/projected/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-api-access-ksp7b\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:45:32.538340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.538337 2574 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:45:32.538340 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.538348 2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/bbce8518-b479-4cdb-a087-e55f6fe77c90-cabundle-cert\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:45:32.538493 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:32.538357 2574 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bbce8518-b479-4cdb-a087-e55f6fe77c90-proxy-tls\") on node \"ip-10-0-132-37.ec2.internal\" DevicePath \"\"" Jul 28 16:45:33.367022 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.366995 2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" Jul 28 16:45:33.367382 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.367023 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64" event={"ID":"bbce8518-b479-4cdb-a087-e55f6fe77c90","Type":"ContainerDied","Data":"4708c25854a6e425dc7a34c1bc1c2a0e4bcca2ddb7c0dff9cc562da15d507e90"} Jul 28 16:45:33.367382 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.367086 2574 scope.go:117] "RemoveContainer" containerID="005f37fe1f7843529248b19314a000f559e045278d382b47ad4a87fd1ee56cea" Jul 28 16:45:33.375218 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.375201 2574 scope.go:117] "RemoveContainer" containerID="c7d6ed59e5f126ad338e02758b1d58ab3246307958f2ad146e2b1619f1c2ca57" Jul 28 16:45:33.382110 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.382094 2574 scope.go:117] "RemoveContainer" containerID="d64edcdb1e38eb8b97f3a82073f6a02ae0b99e8fe7eefd5cc32f3c75e66d40ed" Jul 28 16:45:33.392505 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.392466 2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64"] Jul 28 16:45:33.394158 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.394137 2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-canary-rollback-predictor-5bbb8c8cfc-cxk64"] Jul 28 16:45:33.701155 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.701090 2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" path="/var/lib/kubelet/pods/bbce8518-b479-4cdb-a087-e55f6fe77c90/volumes" Jul 28 16:45:33.935890 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:33.935853 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:45:34.041159 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:34.041117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:45:34.110373 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:34.110348 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:45:42.235864 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:45:42.235827 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:46:58.987815 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:46:58.987764 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:47:00.489154 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:47:00.489117 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:47:09.231189 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:47:09.231156 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:47:09.693640 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:47:09.693551 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:48:44.522947 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:48:44.522916 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:48:44.606650 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:48:44.606616 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:49:02.645754 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:49:02.645701 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:49:02.721582 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:49:02.721540 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:49:07.875430 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:49:07.875398 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:49:07.978284 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:49:07.978259 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:49:17.896676 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:49:17.896644 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:49:23.121408 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:49:23.121373 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:49:23.222085 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:49:23.222053 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:50:43.271945 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:50:43.271905 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:50:48.466301 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:50:48.466268 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:50:48.697423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:50:48.697393 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:51:30.821569 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:51:30.821532 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:51:42.188236 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:51:42.188201 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:08.552362 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:08.552332 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:15.015895 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:15.015849 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:15.116663 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:15.116631 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:44.615780 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:44.615737 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:44.771311 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:44.771276 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:44.871002 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:44.870923 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:47.357919 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:47.357832 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4wfgl_9ab0cb4d-8319-49fc-8031-7e2b24124055/global-pull-secret-syncer/0.log" Jul 28 16:52:47.510920 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:47.510891 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" Jul 28 16:52:47.630045 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:47.629980 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-37.ec2.internal_207f737ba8ca766d2b41e867763749e9/haproxy/0.log" Jul 28 16:52:50.326938 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:50.326902 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:50.426910 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:50.426880 2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jx5jf_90c74a1a-35fb-4b6f-b6b8-ab339710936a/konnectivity-agent/0.log" limit=1024 Jul 28 16:52:51.516997 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.516961 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3db9fe-4bde-44c4-9505-5a185cd365d4/alertmanager/0.log" Jul 28 16:52:51.546655 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.546626 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3db9fe-4bde-44c4-9505-5a185cd365d4/config-reloader/0.log" Jul 28 16:52:51.576638 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.576604 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3db9fe-4bde-44c4-9505-5a185cd365d4/kube-rbac-proxy-web/0.log" Jul 28 16:52:51.604411 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.604384 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3db9fe-4bde-44c4-9505-5a185cd365d4/kube-rbac-proxy/0.log" Jul 28 16:52:51.632239 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.632204 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3db9fe-4bde-44c4-9505-5a185cd365d4/kube-rbac-proxy-metric/0.log" Jul 28 16:52:51.662976 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.662957 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3db9fe-4bde-44c4-9505-5a185cd365d4/prom-label-proxy/0.log" Jul 28 16:52:51.702490 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.702468 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ee3db9fe-4bde-44c4-9505-5a185cd365d4/init-config-reloader/0.log" Jul 28 16:52:51.782844 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.782777 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-n5jsz_db947547-78c7-4d66-b421-46820162100b/kube-state-metrics/0.log" Jul 28 16:52:51.812960 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.812942 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-n5jsz_db947547-78c7-4d66-b421-46820162100b/kube-rbac-proxy-main/0.log" Jul 28 16:52:51.839221 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.839202 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-n5jsz_db947547-78c7-4d66-b421-46820162100b/kube-rbac-proxy-self/0.log" Jul 28 16:52:51.880643 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:51.880621 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-599ccbc4fb-xxcfp_03f55659-aa15-4dfe-b2ed-a32b1d131e2c/metrics-server/0.log" Jul 28 16:52:52.042209 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.042146 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sswf6_2a86c24c-ca67-4414-a724-a27350279c9c/node-exporter/0.log" Jul 28 16:52:52.069842 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.069819 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sswf6_2a86c24c-ca67-4414-a724-a27350279c9c/kube-rbac-proxy/0.log" Jul 28 16:52:52.103839 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.103815 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sswf6_2a86c24c-ca67-4414-a724-a27350279c9c/init-textfile/0.log" Jul 28 16:52:52.229611 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.229581 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-b852l_f9670e41-08a9-4823-88b1-fceaaff248eb/kube-rbac-proxy-main/0.log" Jul 28 16:52:52.262209 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.262182 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-b852l_f9670e41-08a9-4823-88b1-fceaaff248eb/kube-rbac-proxy-self/0.log" Jul 28 16:52:52.289135 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.289115 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-b8f49f5c6-b852l_f9670e41-08a9-4823-88b1-fceaaff248eb/openshift-state-metrics/0.log" Jul 28 16:52:52.619487 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.619463 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-jt2dl_42c0c206-5a4d-42e4-9d45-b0d6b9e9f2a0/prometheus-operator-admission-webhook/0.log" Jul 28 16:52:52.654844 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.654824 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7564b9d65b-rdllx_8e90dda2-820d-4702-8416-eecc8ef92112/telemeter-client/0.log" Jul 28 16:52:52.686887 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.686866 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7564b9d65b-rdllx_8e90dda2-820d-4702-8416-eecc8ef92112/reload/0.log" Jul 28 16:52:52.715803 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:52.715784 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7564b9d65b-rdllx_8e90dda2-820d-4702-8416-eecc8ef92112/kube-rbac-proxy/0.log" Jul 28 16:52:55.183871 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.183842 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69fc7f8776-vb5n2_269faaf4-cbc6-4035-841f-1ce75f8f5b1c/console/0.log" Jul 28 16:52:55.323292 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323256 2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn"] Jul 28 16:52:55.323661 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323629 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" Jul 28 16:52:55.323661 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323654 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" Jul 28 16:52:55.323661 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323670 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="storage-initializer" Jul 28 16:52:55.323990 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323676 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="storage-initializer" Jul 28 16:52:55.323990 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323691 2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kube-rbac-proxy" Jul 28 16:52:55.323990 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323697 2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kube-rbac-proxy" Jul 28 16:52:55.323990 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323778 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kserve-container" Jul 28 16:52:55.323990 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.323786 2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbce8518-b479-4cdb-a087-e55f6fe77c90" containerName="kube-rbac-proxy" Jul 28 16:52:55.327189 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.327168 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.330039 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.330016 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kvj4s\"/\"kube-root-ca.crt\"" Jul 28 16:52:55.331043 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.331022 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kvj4s\"/\"default-dockercfg-nzlck\"" Jul 28 16:52:55.331123 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.331022 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kvj4s\"/\"openshift-service-ca.crt\"" Jul 28 16:52:55.336357 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.336325 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn"] Jul 28 16:52:55.448473 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.448401 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-sys\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.448473 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.448445 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wfd7\" (UniqueName: \"kubernetes.io/projected/ace19907-2009-4093-a666-bf6933ffc718-kube-api-access-2wfd7\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.448630 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.448481 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-podres\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.448630 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.448532 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-lib-modules\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.448630 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.448592 2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-proc\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549069 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549046 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-podres\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549167 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549092 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-lib-modules\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549167 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549131 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-proc\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549271 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549170 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-sys\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549271 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549208 2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wfd7\" (UniqueName: \"kubernetes.io/projected/ace19907-2009-4093-a666-bf6933ffc718-kube-api-access-2wfd7\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549271 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549218 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-podres\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549271 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549234 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-proc\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549277 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-sys\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.549423 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.549298 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ace19907-2009-4093-a666-bf6933ffc718-lib-modules\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.559470 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.559449 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kvj4s\"/\"kube-root-ca.crt\"" Jul 28 16:52:55.570476 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.570457 2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kvj4s\"/\"openshift-service-ca.crt\"" Jul 28 16:52:55.581076 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.581048 2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wfd7\" (UniqueName: \"kubernetes.io/projected/ace19907-2009-4093-a666-bf6933ffc718-kube-api-access-2wfd7\") pod \"perf-node-gather-daemonset-bjprn\" (UID: \"ace19907-2009-4093-a666-bf6933ffc718\") " pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.640514 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.640493 2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kvj4s\"/\"default-dockercfg-nzlck\"" Jul 28 16:52:55.648779 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.648751 2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:55.784082 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:55.784055 2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn"] Jul 28 16:52:55.786503 ip-10-0-132-37 kubenswrapper[2574]: W0728 16:52:55.786472 2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podace19907_2009_4093_a666_bf6933ffc718.slice/crio-beded39068c6dac41c8ebd1bd9c875f0b3a1cfcbf392db83b86b05a634508957 WatchSource:0}: Error finding container beded39068c6dac41c8ebd1bd9c875f0b3a1cfcbf392db83b86b05a634508957: Status 404 returned error can't find the container with id beded39068c6dac41c8ebd1bd9c875f0b3a1cfcbf392db83b86b05a634508957 Jul 28 16:52:56.230774 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:56.230741 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" event={"ID":"ace19907-2009-4093-a666-bf6933ffc718","Type":"ContainerStarted","Data":"168203d9cbdc17ab2295aa748ca3e4b508d9942d03c6c48575b3be78c11ae881"} Jul 28 16:52:56.231137 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:56.230800 2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" event={"ID":"ace19907-2009-4093-a666-bf6933ffc718","Type":"ContainerStarted","Data":"beded39068c6dac41c8ebd1bd9c875f0b3a1cfcbf392db83b86b05a634508957"} Jul 28 16:52:56.231137 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:56.230822 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:52:56.250827 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:56.250782 2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" podStartSLOduration=1.250766095 podStartE2EDuration="1.250766095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 16:52:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 16:52:56.247732145 +0000 UTC m=+1981.106271878" watchObservedRunningTime="2026-07-28 16:52:56.250766095 +0000 UTC m=+1981.109305837" Jul 28 16:52:56.854063 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:56.854038 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r2472_984ba9f4-1fde-4149-a801-56be5ee472db/dns/0.log" Jul 28 16:52:56.878228 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:56.878205 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r2472_984ba9f4-1fde-4149-a801-56be5ee472db/kube-rbac-proxy/0.log" Jul 28 16:52:56.932904 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:56.932885 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rhfmx_27acf1b0-1531-400f-8d62-fd525316311d/dns-node-resolver/0.log" Jul 28 16:52:57.407399 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:57.407369 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-668787855d-fgprt_8bcb2210-dbb0-4034-9781-77ba34f31757/registry/0.log" Jul 28 16:52:57.452897 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:57.452872 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pdvc9_d2db18f9-9353-4501-babc-b9f1e28ab745/node-ca/0.log" Jul 28 16:52:58.594896 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:58.594870 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5s6ln_67ff8958-7dae-4b09-a56e-f001ee724ba2/serve-healthcheck-canary/0.log" Jul 28 16:52:59.031126 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:59.031104 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-95bv8_6ba3bcb8-00f0-478d-bde4-60cbb9c5324d/kube-rbac-proxy/0.log" Jul 28 16:52:59.053168 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:59.053145 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-95bv8_6ba3bcb8-00f0-478d-bde4-60cbb9c5324d/exporter/0.log" Jul 28 16:52:59.075774 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:52:59.075756 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-95bv8_6ba3bcb8-00f0-478d-bde4-60cbb9c5324d/extractor/0.log" Jul 28 16:53:01.344515 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:01.344486 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-85d8b8ddfc-hk4vr_13d4cc11-ed3f-4890-ada9-0f1100dd2de3/manager/0.log" Jul 28 16:53:01.374349 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:01.374324 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-84fc959947-4ldgp_7e5c7dc5-9fc0-4e2f-b9b0-26137b3348d5/manager/0.log" Jul 28 16:53:02.246587 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:02.246482 2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kvj4s/perf-node-gather-daemonset-bjprn" Jul 28 16:53:07.444934 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.444898 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hjhz_40b096ac-3f93-4ef8-a116-b3599aa57561/kube-multus-additional-cni-plugins/0.log" Jul 28 16:53:07.482945 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.482902 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hjhz_40b096ac-3f93-4ef8-a116-b3599aa57561/egress-router-binary-copy/0.log" Jul 28 16:53:07.524746 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.524702 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hjhz_40b096ac-3f93-4ef8-a116-b3599aa57561/cni-plugins/0.log" Jul 28 16:53:07.569615 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.569592 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hjhz_40b096ac-3f93-4ef8-a116-b3599aa57561/bond-cni-plugin/0.log" Jul 28 16:53:07.609797 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.609778 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hjhz_40b096ac-3f93-4ef8-a116-b3599aa57561/routeoverride-cni/0.log" Jul 28 16:53:07.653867 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.653849 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hjhz_40b096ac-3f93-4ef8-a116-b3599aa57561/whereabouts-cni-bincopy/0.log" Jul 28 16:53:07.689785 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.689760 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8hjhz_40b096ac-3f93-4ef8-a116-b3599aa57561/whereabouts-cni/0.log" Jul 28 16:53:07.999037 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:07.999016 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-tbkgq_e678a606-ebd7-4932-997b-12a94ef7f712/kube-multus/0.log" Jul 28 16:53:08.103504 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:08.103487 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mgt4v_7600bbc2-65b3-4c9f-87aa-053bf079816c/network-metrics-daemon/0.log" Jul 28 16:53:08.142797 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:08.142774 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mgt4v_7600bbc2-65b3-4c9f-87aa-053bf079816c/kube-rbac-proxy/0.log" Jul 28 16:53:08.992215 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:08.992186 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/ovn-controller/0.log" Jul 28 16:53:09.032404 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:09.032378 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/ovn-acl-logging/0.log" Jul 28 16:53:09.073865 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:09.073843 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/kube-rbac-proxy-node/0.log" Jul 28 16:53:09.114491 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:09.114471 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 16:53:09.147407 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:09.147385 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/northd/0.log" Jul 28 16:53:09.174876 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:09.174857 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/nbdb/0.log" Jul 28 16:53:09.203872 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:09.203856 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/sbdb/0.log" Jul 28 16:53:09.258853 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:09.258807 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bqh6d_0216df9b-1601-4dfc-b823-c0d00e69ee69/ovnkube-controller/0.log" Jul 28 16:53:10.932745 ip-10-0-132-37 kubenswrapper[2574]: I0728 16:53:10.932703 2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-7hxvb_988926e8-4287-4223-a8b2-8f7d813fedfc/network-check-target-container/0.log"