Jul 08 13:23:16.445219 ip-10-0-129-157 systemd[1]: Starting Kubernetes Kubelet... Jul 08 13:23:16.827518 ip-10-0-129-157 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 13:23:16.827518 ip-10-0-129-157 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 13:23:16.827518 ip-10-0-129-157 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 13:23:16.827518 ip-10-0-129-157 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 13:23:16.827518 ip-10-0-129-157 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 13:23:16.828542 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.828462 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 13:23:16.833468 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833453 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:16.833468 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833467 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833472 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833476 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833479 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833482 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833484 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833487 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833490 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833492 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833495 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833497 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833500 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833502 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833504 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833507 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833509 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833511 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833514 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833517 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833519 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:16.833537 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833522 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833524 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833527 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833534 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833539 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833543 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833546 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833549 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833552 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833555 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833557 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833560 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833562 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833565 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833568 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833571 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833573 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833575 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833578 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:16.834032 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833582 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833584 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833587 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833589 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833592 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833594 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833597 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833599 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833601 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833604 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833606 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833608 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833612 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833615 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833617 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833621 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833623 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833626 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833628 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:16.834518 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833631 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833633 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833636 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833639 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833641 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833643 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833646 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833648 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833651 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833653 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833656 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833658 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833666 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833668 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833674 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833677 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833680 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833683 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833685 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:16.834993 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833688 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833691 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833693 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833696 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833698 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833701 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833703 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833706 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833708 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833712 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833714 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833717 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833719 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833722 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833724 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833726 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833729 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833731 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833734 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833736 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:16.835472 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.833738 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834450 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834457 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834461 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834465 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834469 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834472 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834475 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834478 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834480 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834483 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834485 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834488 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834490 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834493 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834495 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834498 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834501 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834503 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:16.835980 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834505 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834508 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834512 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834517 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834520 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834523 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834525 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834528 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834531 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834534 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834536 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834539 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834541 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834544 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834547 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834550 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834552 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834554 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834558 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:16.836455 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834560 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834563 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834565 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834568 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834572 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834575 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834578 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834581 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834583 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834586 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834589 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834592 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834595 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834597 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834600 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834603 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834606 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834608 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834612 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834615 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:16.836919 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834617 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834620 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834623 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834625 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834628 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834630 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834632 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834635 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834637 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834639 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834642 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834644 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834647 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834649 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834652 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834654 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834657 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834659 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834663 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:16.837481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834665 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834668 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834670 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834673 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834675 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834678 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834680 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834683 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834686 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834689 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834692 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834694 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834696 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834699 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834701 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834704 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834707 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834709 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834711 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834714 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:16.837965 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834716 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834719 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.834721 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835084 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835089 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835092 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835095 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835100 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835104 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835106 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835109 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835112 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835115 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835118 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835120 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835122 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835125 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835128 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835130 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835133 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:16.838449 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835136 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835139 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835141 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835144 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835146 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835148 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835151 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835155 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835158 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835161 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835163 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835166 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835168 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835171 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835173 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835175 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835178 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835180 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835183 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:16.838942 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835185 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835190 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835193 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835195 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835198 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835200 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835204 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835207 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835210 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835213 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835216 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835218 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835222 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835225 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835227 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835230 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835234 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835237 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835240 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:16.839406 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835242 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835245 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835248 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835250 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835252 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835255 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835257 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835260 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835262 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835264 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835267 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835269 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835271 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835274 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835276 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835279 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835281 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835284 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835286 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835289 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:16.839887 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835291 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835294 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835296 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835300 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835302 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835305 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835308 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835310 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835313 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835315 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835318 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835320 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835323 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835325 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835327 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835330 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835332 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835335 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835337 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835339 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:16.840405 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835342 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835344 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835346 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.835349 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835423 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835433 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835439 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835443 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835453 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835457 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835461 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835465 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835468 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835471 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835474 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835477 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835480 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835484 2560 flags.go:64] FLAG: --cgroup-root="" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835486 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835489 2560 flags.go:64] FLAG: --client-ca-file="" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835492 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835495 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835498 2560 flags.go:64] FLAG: --cluster-domain="" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835501 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 13:23:16.840882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835504 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835506 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835509 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835513 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835516 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835519 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835522 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835524 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835527 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835530 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835533 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835535 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835539 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835542 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835545 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835547 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835550 2560 flags.go:64] FLAG: --enable-server="true" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835559 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835565 2560 flags.go:64] FLAG: --event-burst="100" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835568 2560 flags.go:64] FLAG: --event-qps="50" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835571 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835574 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835577 2560 flags.go:64] FLAG: --eviction-hard="" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835580 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 13:23:16.841492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835583 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835588 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835591 2560 flags.go:64] FLAG: --eviction-soft="" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835593 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835596 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835599 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835601 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835604 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835607 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835610 2560 flags.go:64] FLAG: --feature-gates="" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835613 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835616 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835619 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835622 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835627 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835629 2560 flags.go:64] FLAG: --help="false" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835632 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-129-157.ec2.internal" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835635 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835638 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835640 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835643 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835647 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835649 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 13:23:16.842080 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835652 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835654 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835657 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835665 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835668 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835671 2560 flags.go:64] FLAG: --kube-reserved="" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835674 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835676 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835679 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835682 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835685 2560 flags.go:64] FLAG: --lock-file="" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835688 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835691 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835693 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835698 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835701 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835704 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835707 2560 flags.go:64] FLAG: --logging-format="text" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835709 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835712 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835715 2560 flags.go:64] FLAG: --manifest-url="" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835718 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835722 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835726 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835730 2560 flags.go:64] FLAG: --max-pods="110" Jul 08 13:23:16.842703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835733 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835736 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835738 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835741 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835744 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835747 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835749 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835763 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835766 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835769 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835772 2560 flags.go:64] FLAG: --pod-cidr="" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835775 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835780 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835782 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835785 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835788 2560 flags.go:64] FLAG: --port="10250" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835790 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835793 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-067f365ad79434163" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835797 2560 flags.go:64] FLAG: --qos-reserved="" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835800 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835803 2560 flags.go:64] FLAG: --register-node="true" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835806 2560 flags.go:64] FLAG: --register-with-taints="" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835809 2560 flags.go:64] FLAG: --registry-burst="10" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835812 2560 flags.go:64] FLAG: --registry-qps="5" Jul 08 13:23:16.843333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835814 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835817 2560 flags.go:64] FLAG: --reserved-memory="" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835820 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835823 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835826 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835829 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835832 2560 flags.go:64] FLAG: --runonce="false" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835835 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835838 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835840 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835843 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835846 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835849 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835851 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835854 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835856 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835859 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835862 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835864 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835867 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835870 2560 flags.go:64] FLAG: --system-cgroups="" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835873 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835878 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835880 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835883 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 13:23:16.843916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835889 2560 flags.go:64] FLAG: --tls-min-version="" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835893 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835896 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835899 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835916 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835934 2560 flags.go:64] FLAG: --v="2" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835939 2560 flags.go:64] FLAG: --version="false" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835943 2560 flags.go:64] FLAG: --vmodule="" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835946 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.835949 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836058 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836062 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836065 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836069 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836072 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836075 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836077 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836080 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836083 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836085 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836088 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:16.844527 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836090 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836093 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836095 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836098 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836101 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836104 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836108 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836111 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836114 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836117 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836119 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836122 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836126 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836128 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836131 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836133 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836136 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836138 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836141 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836143 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:16.845066 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836146 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836148 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836150 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836153 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836157 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836160 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836162 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836165 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836167 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836171 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836174 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836176 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836179 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836181 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836184 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836186 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836189 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836191 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836194 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:16.845551 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836196 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836199 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836201 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836204 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836206 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836213 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836215 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836218 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836221 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836223 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836226 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836228 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836231 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836233 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836236 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836239 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836241 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836244 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836247 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:16.846117 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836251 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836254 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836257 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836259 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836262 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836265 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836267 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836270 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836273 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836275 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836278 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836280 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836283 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836285 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836288 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836291 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836293 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836296 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836299 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836302 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:16.846617 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836304 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836307 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836309 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836312 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836314 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836317 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836319 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836322 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836324 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.836326 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.837184 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.843641 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.843655 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.843673 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.843679 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 13:23:16.847106 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843731 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843735 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843739 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843742 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843744 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843748 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843750 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843753 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843756 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843759 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843761 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843764 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843767 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843770 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843773 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843775 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843778 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843781 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843783 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:16.847481 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843786 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843788 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843791 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843795 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843799 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843802 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843806 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843809 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843812 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843815 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843819 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843821 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843824 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843826 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843829 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843831 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843834 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843837 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843839 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:16.847937 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843842 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843846 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843849 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843852 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843854 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843857 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843860 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843863 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843866 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843869 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843871 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843874 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843876 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843879 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843882 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843885 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843887 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843889 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843892 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:16.848427 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843894 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843897 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843899 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843902 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843905 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843907 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843910 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843912 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843914 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843917 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843937 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843941 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843944 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843946 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843948 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843951 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843954 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843956 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843959 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843961 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:16.848885 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843964 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843967 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843970 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843972 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843975 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843977 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843979 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843982 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843984 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843986 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843989 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843991 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843994 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843996 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.843998 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844001 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844003 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844006 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844008 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844010 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:16.849393 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844013 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844015 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.844020 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844125 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844129 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844131 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844134 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844136 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844139 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844142 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844144 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844147 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844149 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844152 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844155 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 13:23:16.849908 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844158 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844160 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844163 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844165 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844167 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844170 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844172 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844175 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844177 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844179 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844182 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844184 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844188 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844191 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844194 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844197 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844200 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844204 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844207 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 13:23:16.850338 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844209 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844212 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844214 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844217 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844219 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844221 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844223 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844226 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844228 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844231 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844233 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844235 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844238 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844241 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844243 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844245 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844248 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844250 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844253 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 13:23:16.850790 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844255 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844257 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844260 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844262 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844264 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844267 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844269 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844271 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844275 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844277 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844280 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844283 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844287 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844290 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844292 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844295 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844298 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844300 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844303 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844305 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 13:23:16.851376 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844308 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844311 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844314 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844316 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844319 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844322 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844324 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844327 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844330 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844332 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844334 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844337 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844339 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844341 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844344 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844346 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844348 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844351 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844353 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844355 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 13:23:16.852171 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844358 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844361 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844363 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844366 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844368 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844370 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844373 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844375 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:16.844377 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.844382 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.844467 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.846379 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.847755 2560 server.go:1019] "Starting client certificate rotation" Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.847851 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 13:23:16.852642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.847885 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 13:23:16.853856 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.853838 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 13:23:16.858152 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.858134 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 13:23:16.867353 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.867336 2560 log.go:25] "Validated CRI v1 runtime API" Jul 08 13:23:16.869312 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.869294 2560 log.go:25] "Validated CRI v1 image API" Jul 08 13:23:16.870154 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.870140 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 13:23:16.874771 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.874752 2560 fs.go:135] Filesystem UUIDs: map[3407748c-5d05-4645-a6a4-3e018e0efc26:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 a34d7e31-c9f1-4ea8-a256-b65d6f12907d:/dev/nvme0n1p3] Jul 08 13:23:16.874845 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.874769 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 13:23:16.879711 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.879590 2560 manager.go:216] Machine: {Timestamp:2026-07-08 13:23:16.878338697 +0000 UTC m=+0.323526357 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098877 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bed2cf34beeeab8bf3a2ca9741e81 SystemUUID:ec2bed2c-f34b-eeea-b8bf-3a2ca9741e81 BootID:5b69f52a-948e-496a-9ce1-4ffe1d119111 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f0:81:c5:68:f7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f0:81:c5:68:f7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:aa:9b:0c:be:17 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 13:23:16.879711 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.879701 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 13:23:16.879856 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.879771 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 13:23:16.880262 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.880248 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 13:23:16.882328 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.882304 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 13:23:16.882498 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.882333 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 13:23:16.883524 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.883511 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 13:23:16.883585 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.883528 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 13:23:16.883585 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.883539 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 13:23:16.884114 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.884103 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 13:23:16.885235 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.885224 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 13:23:16.887741 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.887727 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 13:23:16.888044 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888032 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 13:23:16.888119 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888071 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 08 13:23:16.888119 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888086 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 13:23:16.888119 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888105 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 13:23:16.888119 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888118 2560 kubelet.go:398] "Adding apiserver pod source" Jul 08 13:23:16.888275 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888130 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 13:23:16.888539 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888519 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 13:23:16.888622 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.888550 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 13:23:16.893369 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.893344 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 13:23:16.894979 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.894966 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 13:23:16.895030 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.894997 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 13:23:16.896055 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896045 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 13:23:16.896092 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896080 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 13:23:16.896092 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896088 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 13:23:16.896153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896094 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 13:23:16.896153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896107 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 13:23:16.896153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896116 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 13:23:16.896153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896130 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 13:23:16.896153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896139 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 13:23:16.896153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896151 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 13:23:16.896309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896159 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 13:23:16.896309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896180 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 13:23:16.896309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896188 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 13:23:16.896309 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.896268 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 13:23:16.896309 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.896290 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 13:23:16.896697 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896686 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 13:23:16.896728 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.896699 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 13:23:16.899812 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.899795 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gh46s" Jul 08 13:23:16.899999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.899987 2560 server.go:1268] "Started kubelet" Jul 08 13:23:16.900156 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.900105 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 13:23:16.900230 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.900213 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 13:23:16.900278 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.900139 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 13:23:16.900538 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.900509 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 13:23:16.901007 ip-10-0-129-157 systemd[1]: Started Kubernetes Kubelet. Jul 08 13:23:16.902290 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.902271 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 13:23:16.903063 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.903043 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 08 13:23:16.905601 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.905585 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 13:23:16.907211 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.907196 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gh46s" Jul 08 13:23:16.907526 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.907512 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 13:23:16.907567 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.907514 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 13:23:16.907906 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.907757 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:16.908185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908168 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 13:23:16.908185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908182 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 13:23:16.908345 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908252 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 13:23:16.908345 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908302 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 08 13:23:16.908345 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908308 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 08 13:23:16.908456 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908394 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 13:23:16.908456 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908409 2560 factory.go:55] Registering systemd factory Jul 08 13:23:16.908456 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908420 2560 factory.go:223] Registration of the systemd container factory successfully Jul 08 13:23:16.908680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908656 2560 factory.go:153] Registering CRI-O factory Jul 08 13:23:16.908680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908671 2560 factory.go:223] Registration of the crio container factory successfully Jul 08 13:23:16.908788 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908692 2560 factory.go:103] Registering Raw factory Jul 08 13:23:16.908788 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.908705 2560 manager.go:1196] Started watching for new ooms in manager Jul 08 13:23:16.909137 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.909123 2560 manager.go:349] Starting recovery of all containers Jul 08 13:23:16.912766 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.912741 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:16.916276 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.916117 2560 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-157.ec2.internal\" not found" node="ip-10-0-129-157.ec2.internal" Jul 08 13:23:16.921660 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.921645 2560 manager.go:354] Recovery completed Jul 08 13:23:16.923117 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.923091 2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 08 13:23:16.925956 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.925944 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:16.927672 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.927654 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:16.927736 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.927684 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:16.927736 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.927694 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:16.928180 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.928167 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 13:23:16.928215 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.928181 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:16.928215 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.928182 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 13:23:16.928215 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.928201 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 13:23:16.928857 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.928844 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:16.928900 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.928869 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:16.928900 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.928879 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:16.929124 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.929114 2560 policy_none.go:49] "None policy: Start" Jul 08 13:23:16.929162 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.929129 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 13:23:16.929162 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.929141 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 13:23:16.929995 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.929985 2560 policy_none.go:47] "Start" Jul 08 13:23:16.965512 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.965500 2560 manager.go:337] "Starting Device Plugin manager" Jul 08 13:23:16.965583 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.965527 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 13:23:16.965583 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.965536 2560 server.go:87] "Starting device plugin registration server" Jul 08 13:23:16.965738 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.965721 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 13:23:16.965832 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.965741 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 13:23:16.965832 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.965816 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 13:23:16.965913 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.965869 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 13:23:16.965913 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:16.965876 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 13:23:16.966346 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.966328 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 13:23:16.966407 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:16.966375 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.066556 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.066536 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:17.067651 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.067563 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:17.067651 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.067595 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:17.067651 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.067610 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:17.067651 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.067640 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.076692 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.076680 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.076734 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.076700 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-157.ec2.internal\": node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.093548 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.093502 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.122457 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.122429 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 13:23:17.123704 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.123692 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 13:23:17.123762 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.123730 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 13:23:17.123762 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.123749 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 13:23:17.123857 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.123777 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 13:23:17.125951 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.125914 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:17.193959 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.193917 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.224434 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.224412 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal"] Jul 08 13:23:17.224491 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.224474 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:17.225800 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.225785 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:17.225861 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.225813 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:17.225861 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.225823 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:17.227168 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.227157 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:17.227286 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.227272 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.227335 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.227299 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:17.228578 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.228562 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:17.228636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.228588 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:17.228636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.228597 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:17.228695 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.228568 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:17.228695 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.228664 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:17.228695 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.228676 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:17.229774 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.229762 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.229822 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.229785 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 13:23:17.230432 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.230414 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientMemory" Jul 08 13:23:17.230518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.230448 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 13:23:17.230518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.230462 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeHasSufficientPID" Jul 08 13:23:17.253559 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.253537 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-157.ec2.internal\" not found" node="ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.257075 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.257058 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-157.ec2.internal\" not found" node="ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.294372 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.294352 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.311607 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.311586 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2c0eff54d21d6540cf23785ec244815-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"d2c0eff54d21d6540cf23785ec244815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.311673 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.311609 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78255babc9a1b932c1400b78f8b89518-config\") pod \"kube-apiserver-proxy-ip-10-0-129-157.ec2.internal\" (UID: \"78255babc9a1b932c1400b78f8b89518\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.311673 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.311624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2c0eff54d21d6540cf23785ec244815-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"d2c0eff54d21d6540cf23785ec244815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.395033 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.394971 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.412387 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.412368 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2c0eff54d21d6540cf23785ec244815-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"d2c0eff54d21d6540cf23785ec244815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.412455 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.412392 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2c0eff54d21d6540cf23785ec244815-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"d2c0eff54d21d6540cf23785ec244815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.412455 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.412410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78255babc9a1b932c1400b78f8b89518-config\") pod \"kube-apiserver-proxy-ip-10-0-129-157.ec2.internal\" (UID: \"78255babc9a1b932c1400b78f8b89518\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.412455 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.412433 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/78255babc9a1b932c1400b78f8b89518-config\") pod \"kube-apiserver-proxy-ip-10-0-129-157.ec2.internal\" (UID: \"78255babc9a1b932c1400b78f8b89518\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.412568 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.412479 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d2c0eff54d21d6540cf23785ec244815-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"d2c0eff54d21d6540cf23785ec244815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.412568 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.412481 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d2c0eff54d21d6540cf23785ec244815-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal\" (UID: \"d2c0eff54d21d6540cf23785ec244815\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.495528 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.495505 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.556988 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.556952 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.559484 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.559470 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.596061 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.596032 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.696613 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.696546 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-157.ec2.internal\" not found" Jul 08 13:23:17.712261 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.712240 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:17.808137 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.808115 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.820446 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.820428 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 13:23:17.822442 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.822430 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" Jul 08 13:23:17.830703 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.830682 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 13:23:17.846834 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.846812 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 13:23:17.846975 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.846953 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 13:23:17.847022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.846974 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 13:23:17.847022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.846981 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 13:23:17.889169 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.889144 2560 apiserver.go:52] "Watching apiserver" Jul 08 13:23:17.907061 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.906436 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 13:23:17.907636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.907618 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 13:23:17.907725 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.907619 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal","openshift-dns/node-resolver-v2rcg","openshift-image-registry/node-ca-m44xj","openshift-multus/multus-57gt4","openshift-multus/network-metrics-daemon-np9nc","openshift-network-operator/iptables-alerter-wxxnq","openshift-ovn-kubernetes/ovnkube-node-gt94q","kube-system/konnectivity-agent-c2m6r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv","openshift-cluster-node-tuning-operator/tuned-r2cbb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal","openshift-multus/multus-additional-cni-plugins-tz28k","openshift-network-diagnostics/network-check-target-qkjqn","kube-system/global-pull-secret-syncer-ttqmm"] Jul 08 13:23:17.909043 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.908992 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 13:18:17 +0000 UTC" deadline="2027-12-01 17:04:04.846736472 +0000 UTC" Jul 08 13:23:17.909100 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.909048 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12267h40m46.937691686s" Jul 08 13:23:17.909100 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.909027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:17.910250 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.910230 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:17.911319 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.911296 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:17.911540 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.911521 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 13:23:17.911540 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.911534 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5xcqm\"" Jul 08 13:23:17.911646 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.911534 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 13:23:17.912444 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.912430 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57gt4" Jul 08 13:23:17.912627 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.912607 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 13:23:17.912720 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.912624 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kj4bg\"" Jul 08 13:23:17.912720 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.912703 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 13:23:17.913435 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.913415 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lrr5t\"" Jul 08 13:23:17.913558 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.913541 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 13:23:17.913623 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.913610 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 13:23:17.913716 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.913701 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 13:23:17.913791 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.913775 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:17.913871 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.913844 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:17.914571 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.914551 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1589525f-a8f4-4b36-8285-1efbd90f864a-agent-certs\") pod \"konnectivity-agent-c2m6r\" (UID: \"1589525f-a8f4-4b36-8285-1efbd90f864a\") " pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:17.914652 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.914587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1589525f-a8f4-4b36-8285-1efbd90f864a-konnectivity-ca\") pod \"konnectivity-agent-c2m6r\" (UID: \"1589525f-a8f4-4b36-8285-1efbd90f864a\") " pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:17.914735 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.914718 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 13:23:17.914885 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.914850 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:17.915058 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.915037 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 13:23:17.915058 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.915047 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 13:23:17.915236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.915135 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rxggx\"" Jul 08 13:23:17.915410 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.915397 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 13:23:17.916225 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.916212 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:17.917497 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.917478 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:17.917840 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.917817 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pxf8s\"" Jul 08 13:23:17.917950 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.917935 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 13:23:17.918059 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.918044 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 13:23:17.918400 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.918386 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 13:23:17.918649 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.918633 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 13:23:17.918685 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.918656 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 13:23:17.918685 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.918673 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zb85s\"" Jul 08 13:23:17.918928 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.918913 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:17.919724 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.919707 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 13:23:17.920027 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.920013 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 13:23:17.920631 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.920617 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:17.920928 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.920899 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x9qkf\"" Jul 08 13:23:17.920928 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.920914 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 13:23:17.920928 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.920926 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 13:23:17.921151 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.921122 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 13:23:17.921252 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.921229 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 13:23:17.921665 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.921647 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 13:23:17.921897 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.921881 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 13:23:17.922062 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.922044 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:17.922128 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.922107 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:17.922216 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.922201 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 13:23:17.922666 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.922654 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-njqq9\"" Jul 08 13:23:17.923309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.923291 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 13:23:17.923384 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.923311 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 13:23:17.923384 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.923302 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-7qjwb\"" Jul 08 13:23:17.923384 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.923292 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 13:23:17.923744 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.923724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:17.923815 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:17.923797 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:17.950101 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.950047 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xfz25" Jul 08 13:23:17.955873 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:17.955857 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xfz25" Jul 08 13:23:18.009374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.009311 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 13:23:18.015327 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015300 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.015452 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015341 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b42a58a6-b426-475f-ba38-e40d34589fbc-hosts-file\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.015452 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-cni-binary-copy\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.015452 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-hostroot\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.015452 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015427 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-sys-fs\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015453 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-run-ovn-kubernetes\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-os-release\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015538 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1589525f-a8f4-4b36-8285-1efbd90f864a-konnectivity-ca\") pod \"konnectivity-agent-c2m6r\" (UID: \"1589525f-a8f4-4b36-8285-1efbd90f864a\") " pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015558 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-etc-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015610 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysconfig\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015643 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65xdt\" (UniqueName: \"kubernetes.io/projected/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-kube-api-access-65xdt\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.015680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-kubelet-config\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015688 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-cni-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-kubelet\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015729 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-cni-netd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-var-lib-kubelet\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015788 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-cnibin\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015807 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d5990eb-341e-4473-bdd5-73e0100a0abb-host-slash\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csrl5\" (UniqueName: \"kubernetes.io/projected/0d5990eb-341e-4473-bdd5-73e0100a0abb-kube-api-access-csrl5\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015871 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015887 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7p2h\" (UniqueName: \"kubernetes.io/projected/a9385e28-6c83-44f0-aec5-a978ae6e6192-kube-api-access-q7p2h\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015904 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-cni-multus\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015920 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.015985 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-log-socket\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovnkube-script-lib\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016030 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-kubernetes\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016050 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-host\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.016129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016072 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-tuned\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016091 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-multus-certs\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016137 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-device-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-systemd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016194 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1589525f-a8f4-4b36-8285-1efbd90f864a-konnectivity-ca\") pod \"konnectivity-agent-c2m6r\" (UID: \"1589525f-a8f4-4b36-8285-1efbd90f864a\") " pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016213 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-node-log\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016236 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-systemd\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26c9c0ad-a236-49e6-b425-7a867b9be873-host\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016265 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26c9c0ad-a236-49e6-b425-7a867b9be873-serviceca\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016282 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5c9s\" (UniqueName: \"kubernetes.io/projected/26c9c0ad-a236-49e6-b425-7a867b9be873-kube-api-access-w5c9s\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016307 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-conf-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016327 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-etc-kubernetes\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-cni-bin\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvjmd\" (UniqueName: \"kubernetes.io/projected/f8039f33-da31-4086-bf04-ceb18cc54ac5-kube-api-access-zvjmd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016382 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42a58a6-b426-475f-ba38-e40d34589fbc-tmp-dir\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016413 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016439 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-modprobe-d\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.016939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016463 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-lib-modules\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016509 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcz9d\" (UniqueName: \"kubernetes.io/projected/b42a58a6-b426-475f-ba38-e40d34589fbc-kube-api-access-kcz9d\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016543 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-system-cni-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016615 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-os-release\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-k8s-cni-cncf-io\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016664 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-etc-selinux\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-systemd-units\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016716 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovn-node-metrics-cert\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016730 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysctl-d\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016771 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wgq\" (UniqueName: \"kubernetes.io/projected/8e451a5a-2e25-4860-810e-c7d1cd7bf260-kube-api-access-s2wgq\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016801 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-run\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016823 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-sys\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016843 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-dbus\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016867 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-slash\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.017844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016890 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-env-overrides\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-netns\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.016948 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-socket-dir-parent\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-kubelet\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017044 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017071 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-ovn\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017113 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d5990eb-341e-4473-bdd5-73e0100a0abb-iptables-alerter-script\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-cnibin\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-cni-bin\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017203 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-daemon-config\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8k8r\" (UniqueName: \"kubernetes.io/projected/92436f7c-7eee-4802-b7eb-dfe846ed0420-kube-api-access-v8k8r\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfgmw\" (UniqueName: \"kubernetes.io/projected/63478318-c772-44b4-b607-c8f885d265b4-kube-api-access-sfgmw\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017271 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysctl-conf\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017294 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-system-cni-dir\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017319 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1589525f-a8f4-4b36-8285-1efbd90f864a-agent-certs\") pod \"konnectivity-agent-c2m6r\" (UID: \"1589525f-a8f4-4b36-8285-1efbd90f864a\") " pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:18.018699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92436f7c-7eee-4802-b7eb-dfe846ed0420-cni-binary-copy\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.019628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-tmp\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.019628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017530 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovnkube-config\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.019628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-socket-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.019628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017585 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-registration-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.019628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017602 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-run-netns\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.019628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017637 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 13:23:18.019628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.017659 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-var-lib-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.020706 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.020686 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1589525f-a8f4-4b36-8285-1efbd90f864a-agent-certs\") pod \"konnectivity-agent-c2m6r\" (UID: \"1589525f-a8f4-4b36-8285-1efbd90f864a\") " pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:18.108470 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.108440 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c0eff54d21d6540cf23785ec244815.slice/crio-b424311cda098c24b9ab5c81371d7c905351d11adb5e7ea10a40b2d03e69e2f9 WatchSource:0}: Error finding container b424311cda098c24b9ab5c81371d7c905351d11adb5e7ea10a40b2d03e69e2f9: Status 404 returned error can't find the container with id b424311cda098c24b9ab5c81371d7c905351d11adb5e7ea10a40b2d03e69e2f9 Jul 08 13:23:18.109003 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.108983 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78255babc9a1b932c1400b78f8b89518.slice/crio-aa33a67db7b7263dea6ca3330c877ee2eb945fbecb9d7497abe8dae2509763c0 WatchSource:0}: Error finding container aa33a67db7b7263dea6ca3330c877ee2eb945fbecb9d7497abe8dae2509763c0: Status 404 returned error can't find the container with id aa33a67db7b7263dea6ca3330c877ee2eb945fbecb9d7497abe8dae2509763c0 Jul 08 13:23:18.116645 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.116623 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 13:23:18.118544 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118510 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wgq\" (UniqueName: \"kubernetes.io/projected/8e451a5a-2e25-4860-810e-c7d1cd7bf260-kube-api-access-s2wgq\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:18.118624 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-run\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.118624 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-sys\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.118624 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118614 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-dbus\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118637 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-slash\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-env-overrides\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118662 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-run\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-netns\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118720 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-socket-dir-parent\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118729 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-netns\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118734 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-slash\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118749 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-kubelet\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.118783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118788 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-kubelet\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118806 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-ovn\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118806 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-socket-dir-parent\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118830 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118844 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d5990eb-341e-4473-bdd5-73e0100a0abb-iptables-alerter-script\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118869 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-cnibin\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118875 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-ovn\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118890 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-cni-bin\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-daemon-config\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118946 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8k8r\" (UniqueName: \"kubernetes.io/projected/92436f7c-7eee-4802-b7eb-dfe846ed0420-kube-api-access-v8k8r\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118967 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-cnibin\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118989 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-dbus\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.118994 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfgmw\" (UniqueName: \"kubernetes.io/projected/63478318-c772-44b4-b607-c8f885d265b4-kube-api-access-sfgmw\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119051 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-cni-bin\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119063 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysctl-conf\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119094 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-system-cni-dir\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.119227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119123 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119153 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92436f7c-7eee-4802-b7eb-dfe846ed0420-cni-binary-copy\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119180 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-tmp\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119202 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-env-overrides\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovnkube-config\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119261 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-socket-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119280 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-system-cni-dir\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-registration-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-run-netns\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119348 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-var-lib-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b42a58a6-b426-475f-ba38-e40d34589fbc-hosts-file\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-cni-binary-copy\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-hostroot\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119466 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0d5990eb-341e-4473-bdd5-73e0100a0abb-iptables-alerter-script\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119502 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-hostroot\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119522 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-var-lib-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.120214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119523 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-sys\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119534 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysctl-conf\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-sys-fs\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.119581 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119595 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-run-ovn-kubernetes\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119609 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-socket-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-os-release\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119634 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-daemon-config\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119665 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-registration-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119689 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-os-release\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119701 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b42a58a6-b426-475f-ba38-e40d34589fbc-hosts-file\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119710 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-run-netns\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.119639 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret podName:9ed8e9d3-dce6-4a00-99aa-f300ece48f4e nodeName:}" failed. No retries permitted until 2026-07-08 13:23:18.619618233 +0000 UTC m=+2.064805905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret") pod "global-pull-secret-syncer-ttqmm" (UID: "9ed8e9d3-dce6-4a00-99aa-f300ece48f4e") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119748 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-sys-fs\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119753 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovnkube-config\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119763 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119796 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-run-ovn-kubernetes\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.121175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-etc-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysconfig\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119856 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65xdt\" (UniqueName: \"kubernetes.io/projected/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-kube-api-access-65xdt\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119885 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-kubelet-config\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119905 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-etc-openvswitch\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119911 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-cni-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-kubelet\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119975 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-cni-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.119991 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-kubelet\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120059 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-kubelet-config\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120058 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92436f7c-7eee-4802-b7eb-dfe846ed0420-cni-binary-copy\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120077 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysconfig\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-cni-netd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120150 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-var-lib-kubelet\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120179 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-cnibin\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120193 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d5990eb-341e-4473-bdd5-73e0100a0abb-host-slash\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120207 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-var-lib-kubelet\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120210 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-cni-binary-copy\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.122022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120209 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-cni-netd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120214 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csrl5\" (UniqueName: \"kubernetes.io/projected/0d5990eb-341e-4473-bdd5-73e0100a0abb-kube-api-access-csrl5\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120253 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-cnibin\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120273 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0d5990eb-341e-4473-bdd5-73e0100a0abb-host-slash\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120291 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7p2h\" (UniqueName: \"kubernetes.io/projected/a9385e28-6c83-44f0-aec5-a978ae6e6192-kube-api-access-q7p2h\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120432 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9385e28-6c83-44f0-aec5-a978ae6e6192-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-cni-multus\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120507 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-log-socket\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120523 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovnkube-script-lib\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120535 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-var-lib-cni-multus\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-kubernetes\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120593 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-log-socket\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.120611 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120632 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-kubernetes\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120641 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-host\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.122593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-host\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.120661 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs podName:8e451a5a-2e25-4860-810e-c7d1cd7bf260 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:18.6206434 +0000 UTC m=+2.065831071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs") pod "network-metrics-daemon-np9nc" (UID: "8e451a5a-2e25-4860-810e-c7d1cd7bf260") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120683 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-tuned\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120704 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-multus-certs\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-device-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120746 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-systemd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-node-log\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120780 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-systemd\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120802 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26c9c0ad-a236-49e6-b425-7a867b9be873-host\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120826 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26c9c0ad-a236-49e6-b425-7a867b9be873-serviceca\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120838 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-device-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120880 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26c9c0ad-a236-49e6-b425-7a867b9be873-host\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5c9s\" (UniqueName: \"kubernetes.io/projected/26c9c0ad-a236-49e6-b425-7a867b9be873-kube-api-access-w5c9s\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120904 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-run-systemd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120914 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-conf-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120963 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-etc-kubernetes\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120948 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-multus-conf-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121010 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-systemd\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121028 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-etc-kubernetes\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-cni-bin\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121044 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovnkube-script-lib\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120841 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-multus-certs\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121076 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvjmd\" (UniqueName: \"kubernetes.io/projected/f8039f33-da31-4086-bf04-ceb18cc54ac5-kube-api-access-zvjmd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42a58a6-b426-475f-ba38-e40d34589fbc-tmp-dir\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-modprobe-d\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.120853 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-node-log\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121077 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-cni-bin\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-lib-modules\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcz9d\" (UniqueName: \"kubernetes.io/projected/b42a58a6-b426-475f-ba38-e40d34589fbc-kube-api-access-kcz9d\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121231 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121231 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121256 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-system-cni-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121309 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-system-cni-dir\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.123939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-os-release\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121357 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-modprobe-d\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121373 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-k8s-cni-cncf-io\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121399 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-etc-selinux\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121444 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42a58a6-b426-475f-ba38-e40d34589fbc-tmp-dir\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121453 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-systemd-units\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121478 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-lib-modules\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121552 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121529 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-host-run-k8s-cni-cncf-io\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovn-node-metrics-cert\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysctl-d\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121668 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/63478318-c772-44b4-b607-c8f885d265b4-etc-selinux\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121697 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92436f7c-7eee-4802-b7eb-dfe846ed0420-os-release\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121733 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/26c9c0ad-a236-49e6-b425-7a867b9be873-serviceca\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121748 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f8039f33-da31-4086-bf04-ceb18cc54ac5-systemd-units\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.121799 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-sysctl-d\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.122121 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9385e28-6c83-44f0-aec5-a978ae6e6192-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.124451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.122783 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-tmp\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.125024 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.123482 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-etc-tuned\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.125024 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.123978 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f8039f33-da31-4086-bf04-ceb18cc54ac5-ovn-node-metrics-cert\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.127002 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.126954 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" event={"ID":"78255babc9a1b932c1400b78f8b89518","Type":"ContainerStarted","Data":"aa33a67db7b7263dea6ca3330c877ee2eb945fbecb9d7497abe8dae2509763c0"} Jul 08 13:23:18.127930 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.127912 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" event={"ID":"d2c0eff54d21d6540cf23785ec244815","Type":"ContainerStarted","Data":"b424311cda098c24b9ab5c81371d7c905351d11adb5e7ea10a40b2d03e69e2f9"} Jul 08 13:23:18.131308 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.131281 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wgq\" (UniqueName: \"kubernetes.io/projected/8e451a5a-2e25-4860-810e-c7d1cd7bf260-kube-api-access-s2wgq\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:18.131520 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.131496 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:18.131609 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.131532 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:18.131609 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.131555 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9hbxq for pod openshift-network-diagnostics/network-check-target-qkjqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:18.131720 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.131610 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq podName:a9e2da49-584e-408a-8346-adf1aae3c535 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:18.631594625 +0000 UTC m=+2.076782278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9hbxq" (UniqueName: "kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq") pod "network-check-target-qkjqn" (UID: "a9e2da49-584e-408a-8346-adf1aae3c535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:18.133271 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.133242 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5c9s\" (UniqueName: \"kubernetes.io/projected/26c9c0ad-a236-49e6-b425-7a867b9be873-kube-api-access-w5c9s\") pod \"node-ca-m44xj\" (UID: \"26c9c0ad-a236-49e6-b425-7a867b9be873\") " pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.133448 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.133422 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfgmw\" (UniqueName: \"kubernetes.io/projected/63478318-c772-44b4-b607-c8f885d265b4-kube-api-access-sfgmw\") pod \"aws-ebs-csi-driver-node-j9ndv\" (UID: \"63478318-c772-44b4-b607-c8f885d265b4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.133684 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.133663 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7p2h\" (UniqueName: \"kubernetes.io/projected/a9385e28-6c83-44f0-aec5-a978ae6e6192-kube-api-access-q7p2h\") pod \"multus-additional-cni-plugins-tz28k\" (UID: \"a9385e28-6c83-44f0-aec5-a978ae6e6192\") " pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.133779 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.133686 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcz9d\" (UniqueName: \"kubernetes.io/projected/b42a58a6-b426-475f-ba38-e40d34589fbc-kube-api-access-kcz9d\") pod \"node-resolver-v2rcg\" (UID: \"b42a58a6-b426-475f-ba38-e40d34589fbc\") " pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.134038 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.133992 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8k8r\" (UniqueName: \"kubernetes.io/projected/92436f7c-7eee-4802-b7eb-dfe846ed0420-kube-api-access-v8k8r\") pod \"multus-57gt4\" (UID: \"92436f7c-7eee-4802-b7eb-dfe846ed0420\") " pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.134386 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.134365 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csrl5\" (UniqueName: \"kubernetes.io/projected/0d5990eb-341e-4473-bdd5-73e0100a0abb-kube-api-access-csrl5\") pod \"iptables-alerter-wxxnq\" (UID: \"0d5990eb-341e-4473-bdd5-73e0100a0abb\") " pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.134929 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.134914 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvjmd\" (UniqueName: \"kubernetes.io/projected/f8039f33-da31-4086-bf04-ceb18cc54ac5-kube-api-access-zvjmd\") pod \"ovnkube-node-gt94q\" (UID: \"f8039f33-da31-4086-bf04-ceb18cc54ac5\") " pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.135012 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.134954 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65xdt\" (UniqueName: \"kubernetes.io/projected/e1a5f288-78a1-4b5e-a594-27efdd7d1b60-kube-api-access-65xdt\") pod \"tuned-r2cbb\" (UID: \"e1a5f288-78a1-4b5e-a594-27efdd7d1b60\") " pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.238330 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.238254 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:18.246755 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.246734 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1589525f_a8f4_4b36_8285_1efbd90f864a.slice/crio-b4f04ff8c80910d28d39b91e773b33924ce9c59cb1fcc038add6caec7ab9fb15 WatchSource:0}: Error finding container b4f04ff8c80910d28d39b91e773b33924ce9c59cb1fcc038add6caec7ab9fb15: Status 404 returned error can't find the container with id b4f04ff8c80910d28d39b91e773b33924ce9c59cb1fcc038add6caec7ab9fb15 Jul 08 13:23:18.253953 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.253938 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v2rcg" Jul 08 13:23:18.259804 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.259786 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m44xj" Jul 08 13:23:18.260788 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.260770 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42a58a6_b426_475f_ba38_e40d34589fbc.slice/crio-c38fbc104b086ec9f5adfad1f1b45961b827468ea57af8b28906a8d0d795ac47 WatchSource:0}: Error finding container c38fbc104b086ec9f5adfad1f1b45961b827468ea57af8b28906a8d0d795ac47: Status 404 returned error can't find the container with id c38fbc104b086ec9f5adfad1f1b45961b827468ea57af8b28906a8d0d795ac47 Jul 08 13:23:18.267823 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.267804 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26c9c0ad_a236_49e6_b425_7a867b9be873.slice/crio-0b36b445dec785fabf3bbc54d217ebecbb79b829b1ff23b1a2d3f70f565ea86b WatchSource:0}: Error finding container 0b36b445dec785fabf3bbc54d217ebecbb79b829b1ff23b1a2d3f70f565ea86b: Status 404 returned error can't find the container with id 0b36b445dec785fabf3bbc54d217ebecbb79b829b1ff23b1a2d3f70f565ea86b Jul 08 13:23:18.276015 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.275985 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-57gt4" Jul 08 13:23:18.306275 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.306243 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tz28k" Jul 08 13:23:18.313068 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.313048 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9385e28_6c83_44f0_aec5_a978ae6e6192.slice/crio-ee9054e279d28a14f9e8669698dbff29917fdc4c39c52a8268c1c5ce2cf9dd2f WatchSource:0}: Error finding container ee9054e279d28a14f9e8669698dbff29917fdc4c39c52a8268c1c5ce2cf9dd2f: Status 404 returned error can't find the container with id ee9054e279d28a14f9e8669698dbff29917fdc4c39c52a8268c1c5ce2cf9dd2f Jul 08 13:23:18.318111 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.318093 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" Jul 08 13:23:18.325524 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.325504 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63478318_c772_44b4_b607_c8f885d265b4.slice/crio-5ec86ac0ee4f0d4f173f5ce2f8bf963036409ed70922905d8afb1d677b0f9e9c WatchSource:0}: Error finding container 5ec86ac0ee4f0d4f173f5ce2f8bf963036409ed70922905d8afb1d677b0f9e9c: Status 404 returned error can't find the container with id 5ec86ac0ee4f0d4f173f5ce2f8bf963036409ed70922905d8afb1d677b0f9e9c Jul 08 13:23:18.336234 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.336215 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:18.345882 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.345860 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8039f33_da31_4086_bf04_ceb18cc54ac5.slice/crio-06decc41bd5505975fd82796494f674a15d212c87d89ad9655580401d8c6e85d WatchSource:0}: Error finding container 06decc41bd5505975fd82796494f674a15d212c87d89ad9655580401d8c6e85d: Status 404 returned error can't find the container with id 06decc41bd5505975fd82796494f674a15d212c87d89ad9655580401d8c6e85d Jul 08 13:23:18.351699 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.351683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" Jul 08 13:23:18.358150 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.358134 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wxxnq" Jul 08 13:23:18.359148 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.359128 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a5f288_78a1_4b5e_a594_27efdd7d1b60.slice/crio-378fd97f0ff07b6693c9e15f9d6aa5b0872fca06ff6fa29b269c578c1608ce0d WatchSource:0}: Error finding container 378fd97f0ff07b6693c9e15f9d6aa5b0872fca06ff6fa29b269c578c1608ce0d: Status 404 returned error can't find the container with id 378fd97f0ff07b6693c9e15f9d6aa5b0872fca06ff6fa29b269c578c1608ce0d Jul 08 13:23:18.365938 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:18.365916 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d5990eb_341e_4473_bdd5_73e0100a0abb.slice/crio-308136ba4a2e5f01c7d838808ad9e864d47d4287de8fc25549ecfbcb4d83bb04 WatchSource:0}: Error finding container 308136ba4a2e5f01c7d838808ad9e864d47d4287de8fc25549ecfbcb4d83bb04: Status 404 returned error can't find the container with id 308136ba4a2e5f01c7d838808ad9e864d47d4287de8fc25549ecfbcb4d83bb04 Jul 08 13:23:18.470366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.470330 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:18.625016 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.624981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:18.625246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.625041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:18.625246 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.625175 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:18.625377 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.625249 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs podName:8e451a5a-2e25-4860-810e-c7d1cd7bf260 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:19.625228468 +0000 UTC m=+3.070416137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs") pod "network-metrics-daemon-np9nc" (UID: "8e451a5a-2e25-4860-810e-c7d1cd7bf260") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:18.625652 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.625548 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:18.625652 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.625629 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret podName:9ed8e9d3-dce6-4a00-99aa-f300ece48f4e nodeName:}" failed. No retries permitted until 2026-07-08 13:23:19.625608949 +0000 UTC m=+3.070796605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret") pod "global-pull-secret-syncer-ttqmm" (UID: "9ed8e9d3-dce6-4a00-99aa-f300ece48f4e") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:18.666975 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.666930 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:18.725906 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.725804 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:18.726093 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.725984 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:18.726093 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.726009 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:18.726093 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.726021 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9hbxq for pod openshift-network-diagnostics/network-check-target-qkjqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:18.726093 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:18.726079 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq podName:a9e2da49-584e-408a-8346-adf1aae3c535 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:19.726060365 +0000 UTC m=+3.171248033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hbxq" (UniqueName: "kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq") pod "network-check-target-qkjqn" (UID: "a9e2da49-584e-408a-8346-adf1aae3c535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:18.958261 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.958086 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 13:18:18 +0000 UTC" deadline="2027-12-10 23:28:15.027910668 +0000 UTC" Jul 08 13:23:18.958261 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:18.958151 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12490h4m56.069764705s" Jul 08 13:23:19.060656 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.060447 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:19.127185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.126665 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:19.127185 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.126812 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:19.149810 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.149696 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v2rcg" event={"ID":"b42a58a6-b426-475f-ba38-e40d34589fbc","Type":"ContainerStarted","Data":"c38fbc104b086ec9f5adfad1f1b45961b827468ea57af8b28906a8d0d795ac47"} Jul 08 13:23:19.152802 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.152731 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2m6r" event={"ID":"1589525f-a8f4-4b36-8285-1efbd90f864a","Type":"ContainerStarted","Data":"b4f04ff8c80910d28d39b91e773b33924ce9c59cb1fcc038add6caec7ab9fb15"} Jul 08 13:23:19.156858 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.156778 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wxxnq" event={"ID":"0d5990eb-341e-4473-bdd5-73e0100a0abb","Type":"ContainerStarted","Data":"308136ba4a2e5f01c7d838808ad9e864d47d4287de8fc25549ecfbcb4d83bb04"} Jul 08 13:23:19.162365 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.162253 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"06decc41bd5505975fd82796494f674a15d212c87d89ad9655580401d8c6e85d"} Jul 08 13:23:19.166824 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.166754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" event={"ID":"63478318-c772-44b4-b607-c8f885d265b4","Type":"ContainerStarted","Data":"5ec86ac0ee4f0d4f173f5ce2f8bf963036409ed70922905d8afb1d677b0f9e9c"} Jul 08 13:23:19.181336 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.181274 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerStarted","Data":"ee9054e279d28a14f9e8669698dbff29917fdc4c39c52a8268c1c5ce2cf9dd2f"} Jul 08 13:23:19.187404 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.187368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57gt4" event={"ID":"92436f7c-7eee-4802-b7eb-dfe846ed0420","Type":"ContainerStarted","Data":"f6e8f447f2b0c9b039764486bf1bf98ec721d61893d35194b635d1cd58bda012"} Jul 08 13:23:19.191403 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.191375 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" event={"ID":"e1a5f288-78a1-4b5e-a594-27efdd7d1b60","Type":"ContainerStarted","Data":"378fd97f0ff07b6693c9e15f9d6aa5b0872fca06ff6fa29b269c578c1608ce0d"} Jul 08 13:23:19.197045 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.197018 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m44xj" event={"ID":"26c9c0ad-a236-49e6-b425-7a867b9be873","Type":"ContainerStarted","Data":"0b36b445dec785fabf3bbc54d217ebecbb79b829b1ff23b1a2d3f70f565ea86b"} Jul 08 13:23:19.633939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.633362 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:19.633939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.633653 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:19.633939 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.633558 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:19.633939 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.633774 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret podName:9ed8e9d3-dce6-4a00-99aa-f300ece48f4e nodeName:}" failed. No retries permitted until 2026-07-08 13:23:21.633749895 +0000 UTC m=+5.078937564 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret") pod "global-pull-secret-syncer-ttqmm" (UID: "9ed8e9d3-dce6-4a00-99aa-f300ece48f4e") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:19.633939 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.633785 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:19.633939 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.633839 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs podName:8e451a5a-2e25-4860-810e-c7d1cd7bf260 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:21.633822694 +0000 UTC m=+5.079010361 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs") pod "network-metrics-daemon-np9nc" (UID: "8e451a5a-2e25-4860-810e-c7d1cd7bf260") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:19.735111 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.735073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:19.735292 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.735261 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:19.735292 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.735280 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:19.735292 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.735292 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9hbxq for pod openshift-network-diagnostics/network-check-target-qkjqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:19.735457 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:19.735352 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq podName:a9e2da49-584e-408a-8346-adf1aae3c535 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:21.735333163 +0000 UTC m=+5.180520817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hbxq" (UniqueName: "kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq") pod "network-check-target-qkjqn" (UID: "a9e2da49-584e-408a-8346-adf1aae3c535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:19.958931 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.958837 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 13:18:18 +0000 UTC" deadline="2028-02-14 19:13:16.538626159 +0000 UTC" Jul 08 13:23:19.958931 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:19.958881 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14069h49m56.579749286s" Jul 08 13:23:20.125268 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:20.125232 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:20.125437 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:20.125243 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:20.125437 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:20.125374 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:20.125565 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:20.125439 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:20.472894 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:20.472865 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 13:23:21.125121 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:21.125089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:21.125590 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.125249 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:21.656260 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:21.655304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:21.656260 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:21.655372 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:21.656260 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.655638 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:21.656260 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.655715 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret podName:9ed8e9d3-dce6-4a00-99aa-f300ece48f4e nodeName:}" failed. No retries permitted until 2026-07-08 13:23:25.655691429 +0000 UTC m=+9.100879083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret") pod "global-pull-secret-syncer-ttqmm" (UID: "9ed8e9d3-dce6-4a00-99aa-f300ece48f4e") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:21.656260 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.656156 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:21.656260 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.656210 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs podName:8e451a5a-2e25-4860-810e-c7d1cd7bf260 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:25.656193586 +0000 UTC m=+9.101381253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs") pod "network-metrics-daemon-np9nc" (UID: "8e451a5a-2e25-4860-810e-c7d1cd7bf260") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:21.756958 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:21.756310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:21.756958 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.756501 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:21.756958 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.756525 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:21.756958 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.756540 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9hbxq for pod openshift-network-diagnostics/network-check-target-qkjqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:21.756958 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:21.756600 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq podName:a9e2da49-584e-408a-8346-adf1aae3c535 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:25.756581478 +0000 UTC m=+9.201769131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hbxq" (UniqueName: "kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq") pod "network-check-target-qkjqn" (UID: "a9e2da49-584e-408a-8346-adf1aae3c535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:22.124060 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:22.124025 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:22.124223 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:22.124176 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:22.124223 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:22.124027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:22.124338 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:22.124279 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:23.124519 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:23.124482 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:23.125025 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:23.124639 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:24.124474 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:24.124388 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:24.124474 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:24.124424 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:24.125069 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:24.124620 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:24.125069 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:24.124753 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:25.127336 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:25.126807 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:25.127336 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.126949 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:25.693535 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:25.693495 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:25.693734 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:25.693601 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:25.693734 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.693619 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:25.693734 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.693699 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs podName:8e451a5a-2e25-4860-810e-c7d1cd7bf260 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:33.693677977 +0000 UTC m=+17.138865646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs") pod "network-metrics-daemon-np9nc" (UID: "8e451a5a-2e25-4860-810e-c7d1cd7bf260") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:25.693734 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.693723 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:25.693964 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.693773 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret podName:9ed8e9d3-dce6-4a00-99aa-f300ece48f4e nodeName:}" failed. No retries permitted until 2026-07-08 13:23:33.693759262 +0000 UTC m=+17.138946940 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret") pod "global-pull-secret-syncer-ttqmm" (UID: "9ed8e9d3-dce6-4a00-99aa-f300ece48f4e") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:25.794523 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:25.794480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:25.794704 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.794657 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:25.794704 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.794686 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:25.794704 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.794700 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9hbxq for pod openshift-network-diagnostics/network-check-target-qkjqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:25.794864 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:25.794769 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq podName:a9e2da49-584e-408a-8346-adf1aae3c535 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:33.794745601 +0000 UTC m=+17.239933269 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hbxq" (UniqueName: "kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq") pod "network-check-target-qkjqn" (UID: "a9e2da49-584e-408a-8346-adf1aae3c535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:26.124359 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:26.124327 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:26.124531 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:26.124322 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:26.124531 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:26.124473 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:26.124531 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:26.124521 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:27.125366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:27.125328 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:27.125715 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:27.125467 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:28.124297 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:28.124264 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:28.124494 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:28.124387 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:28.124820 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:28.124691 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:28.124820 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:28.124779 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:29.124315 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:29.124264 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:29.124782 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:29.124424 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:30.124297 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:30.124266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:30.124468 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:30.124266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:30.124468 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:30.124388 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:30.124821 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:30.124494 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:31.124514 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:31.124477 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:31.124908 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:31.124626 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:32.124372 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:32.124335 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:32.124567 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:32.124335 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:32.124567 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:32.124463 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:32.124567 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:32.124545 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:33.124724 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:33.124687 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:33.125210 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.124841 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:33.757681 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:33.757646 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:33.757913 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:33.757745 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:33.757913 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.757816 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:33.757913 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.757873 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:33.757913 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.757891 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs podName:8e451a5a-2e25-4860-810e-c7d1cd7bf260 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:49.757874822 +0000 UTC m=+33.203062489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs") pod "network-metrics-daemon-np9nc" (UID: "8e451a5a-2e25-4860-810e-c7d1cd7bf260") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:33.758146 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.757955 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret podName:9ed8e9d3-dce6-4a00-99aa-f300ece48f4e nodeName:}" failed. No retries permitted until 2026-07-08 13:23:49.757918374 +0000 UTC m=+33.203106024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret") pod "global-pull-secret-syncer-ttqmm" (UID: "9ed8e9d3-dce6-4a00-99aa-f300ece48f4e") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:33.858978 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:33.858947 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:33.859171 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.859150 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:33.859219 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.859176 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:33.859219 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.859190 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9hbxq for pod openshift-network-diagnostics/network-check-target-qkjqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:33.859293 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:33.859258 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq podName:a9e2da49-584e-408a-8346-adf1aae3c535 nodeName:}" failed. No retries permitted until 2026-07-08 13:23:49.859239005 +0000 UTC m=+33.304426668 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hbxq" (UniqueName: "kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq") pod "network-check-target-qkjqn" (UID: "a9e2da49-584e-408a-8346-adf1aae3c535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:34.124234 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:34.124200 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:34.124407 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:34.124237 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:34.124407 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:34.124318 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:34.124407 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:34.124401 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:35.124057 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:35.124027 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:35.124420 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:35.124140 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:36.124896 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:36.124862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:36.125295 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:36.124863 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:36.125295 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:36.125002 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:36.125295 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:36.125066 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:37.125716 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:37.125668 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:37.126172 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:37.125832 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:38.124004 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:38.123972 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:38.124185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:38.123973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:38.124185 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:38.124095 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:38.124185 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:38.124160 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:39.124940 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:39.124889 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:39.125344 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:39.125050 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:40.124801 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.124566 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:40.125029 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.124761 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:40.125029 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:40.124880 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:40.125029 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:40.124915 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:40.246388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.246316 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" event={"ID":"e1a5f288-78a1-4b5e-a594-27efdd7d1b60","Type":"ContainerStarted","Data":"94337c153c8e7a03941e59d815c0792166663ce003bc60f91fcafb279ae07ee6"} Jul 08 13:23:40.247683 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.247654 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" event={"ID":"78255babc9a1b932c1400b78f8b89518","Type":"ContainerStarted","Data":"e3e3fef350c8c94709a59d3ce0ac475aa78073b7998a89a6bbc96e15f192646c"} Jul 08 13:23:40.252332 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252314 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:23:40.252684 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252660 2560 generic.go:356] "Generic (PLEG): container finished" podID="f8039f33-da31-4086-bf04-ceb18cc54ac5" containerID="a00377cdf409e72c7c13ac93a6b6580ac8e1c4a138fcddf8f56b51a6f64fce8a" exitCode=1 Jul 08 13:23:40.252774 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252722 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"dbba4f19dc34c45a977f292f93dbceae4800a7528005565cb424a628b836314b"} Jul 08 13:23:40.252821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252782 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"fde1efaff1201d4cf4e2fee18553c40aea56a2428ad002551ef73eca6b6a9f4f"} Jul 08 13:23:40.252821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252814 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"67170d5bfe50d8b4e4983add3a38bf40a7c99546046993cb9db8930b97035d99"} Jul 08 13:23:40.252903 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"c265e8a5bbf1c35d133b33f28bf30a90ffc5b9cc1e47d23cdf0c7a61221fef4c"} Jul 08 13:23:40.252903 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252892 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerDied","Data":"a00377cdf409e72c7c13ac93a6b6580ac8e1c4a138fcddf8f56b51a6f64fce8a"} Jul 08 13:23:40.253019 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.252964 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"9588b79b88403d8d08e7864fa316aaa61de36a99c137323721f62cc6adfa36f3"} Jul 08 13:23:40.254186 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.254166 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-57gt4" event={"ID":"92436f7c-7eee-4802-b7eb-dfe846ed0420","Type":"ContainerStarted","Data":"52ca12c137e6d611e332978ca451ac4b5e43b6bff7f15ffd759507e3ed2652e3"} Jul 08 13:23:40.264279 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.264240 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r2cbb" podStartSLOduration=1.412588446 podStartE2EDuration="22.264227502s" totalImagesPullingTime="20.851639056s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:40.263846884 +0000 UTC m=+23.709034553" watchObservedRunningTime="2026-07-08 13:23:40.264227502 +0000 UTC m=+23.709415173" Jul 08 13:23:40.292543 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.292506 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-157.ec2.internal" podStartSLOduration=22.2924931 podStartE2EDuration="22.2924931s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:40.276963315 +0000 UTC m=+23.722150984" watchObservedRunningTime="2026-07-08 13:23:40.2924931 +0000 UTC m=+23.737680770" Jul 08 13:23:40.292657 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:40.292613 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-57gt4" podStartSLOduration=2.083446859 podStartE2EDuration="23.292609534s" totalImagesPullingTime="21.209162675s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:40.292272012 +0000 UTC m=+23.737459698" watchObservedRunningTime="2026-07-08 13:23:40.292609534 +0000 UTC m=+23.737797200" Jul 08 13:23:41.124817 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.124616 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:41.125048 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:41.124916 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:41.257896 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.257856 2560 generic.go:356] "Generic (PLEG): container finished" podID="d2c0eff54d21d6540cf23785ec244815" containerID="666100246f8f92d392919fd5e0d45283e8b6b7912456e196bb0ad1e496f8996b" exitCode=0 Jul 08 13:23:41.258096 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.257949 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" event={"ID":"d2c0eff54d21d6540cf23785ec244815","Type":"ContainerDied","Data":"666100246f8f92d392919fd5e0d45283e8b6b7912456e196bb0ad1e496f8996b"} Jul 08 13:23:41.259499 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.259477 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wxxnq" event={"ID":"0d5990eb-341e-4473-bdd5-73e0100a0abb","Type":"ContainerStarted","Data":"5e638fbc2522d213f5022adb2db738344f270903a071f9e12a2180a2aa13f378"} Jul 08 13:23:41.261014 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.260984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" event={"ID":"63478318-c772-44b4-b607-c8f885d265b4","Type":"ContainerStarted","Data":"07dc820e3efae3b06f14feb6c204a0c03f4e5451b449296e09c549cf13951cf0"} Jul 08 13:23:41.262443 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.262421 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9385e28-6c83-44f0-aec5-a978ae6e6192" containerID="ac76324b3294825d342c1d97b656202faa07a511195bea5d2449e3e3693439b2" exitCode=0 Jul 08 13:23:41.262536 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.262485 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerDied","Data":"ac76324b3294825d342c1d97b656202faa07a511195bea5d2449e3e3693439b2"} Jul 08 13:23:41.263952 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.263906 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m44xj" event={"ID":"26c9c0ad-a236-49e6-b425-7a867b9be873","Type":"ContainerStarted","Data":"c5e50f4dc219423b3624db14dad82423480b7c7af6ae67ecb70b3cc58b69190a"} Jul 08 13:23:41.269264 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.269235 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v2rcg" event={"ID":"b42a58a6-b426-475f-ba38-e40d34589fbc","Type":"ContainerStarted","Data":"12554b889df9d3f42863f68d595fd4b1dc7f1cb530a07957ccbcee621683fc92"} Jul 08 13:23:41.271567 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.271545 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-c2m6r" event={"ID":"1589525f-a8f4-4b36-8285-1efbd90f864a","Type":"ContainerStarted","Data":"10e72bb7936059e67a1fa7e1660612182b76a17d096bd99b7fb61792e8b46230"} Jul 08 13:23:41.298424 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.298381 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v2rcg" podStartSLOduration=3.344591932 podStartE2EDuration="24.298363828s" totalImagesPullingTime="20.953771896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:41.285669703 +0000 UTC m=+24.730857374" watchObservedRunningTime="2026-07-08 13:23:41.298363828 +0000 UTC m=+24.743551500" Jul 08 13:23:41.312325 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.312283 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m44xj" podStartSLOduration=3.371154866 podStartE2EDuration="24.312266951s" totalImagesPullingTime="20.941112085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:41.298676116 +0000 UTC m=+24.743863787" watchObservedRunningTime="2026-07-08 13:23:41.312266951 +0000 UTC m=+24.757454621" Jul 08 13:23:41.312536 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.312506 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wxxnq" podStartSLOduration=2.469026915 podStartE2EDuration="23.312497791s" totalImagesPullingTime="20.843470876s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:41.31236296 +0000 UTC m=+24.757550631" watchObservedRunningTime="2026-07-08 13:23:41.312497791 +0000 UTC m=+24.757685462" Jul 08 13:23:41.349631 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.349576 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-c2m6r" podStartSLOduration=3.386876298 podStartE2EDuration="24.349557028s" totalImagesPullingTime="20.96268073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:41.34942881 +0000 UTC m=+24.794616479" watchObservedRunningTime="2026-07-08 13:23:41.349557028 +0000 UTC m=+24.794744701" Jul 08 13:23:41.716315 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.716245 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:41.892981 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.892952 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:41.935320 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.935298 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 13:23:41.958085 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.958060 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:41.980172 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.980061 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T13:23:41.935315339Z","UUID":"63580ccc-2593-48e6-b6a9-2275c66295fb","Handler":null,"Name":"","Endpoint":""} Jul 08 13:23:41.981408 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.981392 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 13:23:41.981489 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:41.981425 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 13:23:42.103567 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:42.103533 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:42.125124 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:42.125055 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:42.125124 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:42.125087 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:42.125620 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:42.125264 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:42.125620 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:42.125305 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:42.275365 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:42.275328 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" event={"ID":"d2c0eff54d21d6540cf23785ec244815","Type":"ContainerStarted","Data":"91155fc6d3ec8a2e0c2bdf52c1d5ac75daa2ac382f5ee7fcb5c0218b4e736bc1"} Jul 08 13:23:42.277191 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:42.277164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" event={"ID":"63478318-c772-44b4-b607-c8f885d265b4","Type":"ContainerStarted","Data":"bef0d6e8c69699bc38b584809e5d2584c54c1187c3ee8eb1eb0780628241d045"} Jul 08 13:23:42.289582 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:42.289538 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-157.ec2.internal" podStartSLOduration=24.28952651 podStartE2EDuration="24.28952651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:42.289193858 +0000 UTC m=+25.734381530" watchObservedRunningTime="2026-07-08 13:23:42.28952651 +0000 UTC m=+25.734714177" Jul 08 13:23:42.493220 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:42.493107 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:43.093629 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.093421 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:43.128610 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.128585 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:43.129065 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:43.128699 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:43.283469 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.283391 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:23:43.283956 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.283902 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"e982f39ae99cdbf934acc357ff27e70988388f4759e0f7d1dabe9143d482115b"} Jul 08 13:23:43.286249 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.286221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" event={"ID":"63478318-c772-44b4-b607-c8f885d265b4","Type":"ContainerStarted","Data":"8739a9c43855f08782f0e201a790ea48ac37ff9f12e8348a2809144e1c58c35e"} Jul 08 13:23:43.293095 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.293067 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:43.303287 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.303247 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j9ndv" podStartSLOduration=3.236459728 podStartE2EDuration="26.303231423s" totalImagesPullingTime="23.066771695s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:43.303129468 +0000 UTC m=+26.748317137" watchObservedRunningTime="2026-07-08 13:23:43.303231423 +0000 UTC m=+26.748419093" Jul 08 13:23:43.491337 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:43.491301 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:44.124899 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:44.124867 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:44.125073 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:44.124994 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:44.125073 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:44.125046 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:44.125143 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:44.125130 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:45.124524 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:45.124281 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:45.125174 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:45.124647 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:45.253985 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:45.253951 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:45.254631 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:45.254615 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:45.292747 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:45.292723 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:23:45.293086 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:45.293055 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"7ba5d7f5c775e3c1cd0de3fbae841b8d8a8c2bf7e4dde46455ccd1666e5a53e3"} Jul 08 13:23:45.293561 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:45.293546 2560 scope.go:117] "RemoveContainer" containerID="a00377cdf409e72c7c13ac93a6b6580ac8e1c4a138fcddf8f56b51a6f64fce8a" Jul 08 13:23:45.293731 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:45.293715 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-c2m6r" Jul 08 13:23:46.124340 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.124312 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:46.124340 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.124343 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:46.124914 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:46.124429 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:46.124914 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:46.124545 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:46.299959 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.299911 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:23:46.300489 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.300455 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" event={"ID":"f8039f33-da31-4086-bf04-ceb18cc54ac5","Type":"ContainerStarted","Data":"b3dc5442862a2dcd4c527a6e9800c3905471aab779e1229ed42a0dffdc979664"} Jul 08 13:23:46.300622 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.300589 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:46.300751 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.300730 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:46.300832 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.300817 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:46.317558 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.317530 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:46.317693 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.317641 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:23:46.327738 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:46.327686 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" podStartSLOduration=7.451121992 podStartE2EDuration="28.327666127s" totalImagesPullingTime="20.876544135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:23:46.326945295 +0000 UTC m=+29.772132963" watchObservedRunningTime="2026-07-08 13:23:46.327666127 +0000 UTC m=+29.772853801" Jul 08 13:23:47.125452 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:47.125417 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:47.125908 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:47.125502 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:48.124947 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:48.124888 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:48.124947 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:48.124888 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:48.125166 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:48.125020 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:48.125166 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:48.125129 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:48.306791 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:48.306762 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9385e28-6c83-44f0-aec5-a978ae6e6192" containerID="9caec89edd5a682ff623df3e0552b1b84b393222e8c0c684f206ab222a888f6a" exitCode=0 Jul 08 13:23:48.307234 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:48.306825 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerDied","Data":"9caec89edd5a682ff623df3e0552b1b84b393222e8c0c684f206ab222a888f6a"} Jul 08 13:23:49.124289 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.124253 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:49.124475 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.124389 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:49.660175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.660142 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:49.770831 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.770773 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:49.780858 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.780828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:49.780998 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.780864 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:49.780998 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.780918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:49.780998 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.780946 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret podName:9ed8e9d3-dce6-4a00-99aa-f300ece48f4e nodeName:}" failed. No retries permitted until 2026-07-08 13:24:21.780910277 +0000 UTC m=+65.226097925 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret") pod "global-pull-secret-syncer-ttqmm" (UID: "9ed8e9d3-dce6-4a00-99aa-f300ece48f4e") : object "kube-system"/"original-pull-secret" not registered Jul 08 13:23:49.781175 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.781010 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:49.781175 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.781060 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs podName:8e451a5a-2e25-4860-810e-c7d1cd7bf260 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:21.781043584 +0000 UTC m=+65.226231254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs") pod "network-metrics-daemon-np9nc" (UID: "8e451a5a-2e25-4860-810e-c7d1cd7bf260") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 13:23:49.870196 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.870171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:49.881454 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.881428 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:49.881564 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.881552 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 13:23:49.881605 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.881570 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 13:23:49.881605 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.881579 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9hbxq for pod openshift-network-diagnostics/network-check-target-qkjqn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:49.881663 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:49.881622 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq podName:a9e2da49-584e-408a-8346-adf1aae3c535 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:21.881609069 +0000 UTC m=+65.326796721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9hbxq" (UniqueName: "kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq") pod "network-check-target-qkjqn" (UID: "a9e2da49-584e-408a-8346-adf1aae3c535") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 13:23:49.965464 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:49.965388 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:50.064456 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.064422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:50.124568 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.124539 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:50.124697 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.124571 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:50.124697 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:50.124659 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:50.124795 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:50.124776 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:50.129064 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.129041 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:50.299023 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.298939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:50.313491 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.313459 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9385e28-6c83-44f0-aec5-a978ae6e6192" containerID="9643cc26aff9e701243a086fa8c79058041862e819e815fdf1c6933acfb155db" exitCode=0 Jul 08 13:23:50.313603 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.313505 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerDied","Data":"9643cc26aff9e701243a086fa8c79058041862e819e815fdf1c6933acfb155db"} Jul 08 13:23:50.674158 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.674120 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:50.761665 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:50.761629 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:51.082299 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.082253 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:51.127955 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.125692 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:51.127955 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:51.126361 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:51.305294 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.305116 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:51.320580 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.320498 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9385e28-6c83-44f0-aec5-a978ae6e6192" containerID="5951a1c36463e19cfdc251c17e5436be8bdf38c2f83a3ba2629767b71862682c" exitCode=0 Jul 08 13:23:51.320580 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.320542 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerDied","Data":"5951a1c36463e19cfdc251c17e5436be8bdf38c2f83a3ba2629767b71862682c"} Jul 08 13:23:51.361366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.361343 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:51.389724 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.389692 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ttqmm"] Jul 08 13:23:51.389853 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.389802 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:51.389942 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:51.389905 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:51.395734 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.395709 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qkjqn"] Jul 08 13:23:51.395903 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.395808 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:51.395903 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:51.395888 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:51.397830 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.397805 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-np9nc"] Jul 08 13:23:51.397958 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.397905 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:51.398069 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:51.398013 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:51.485884 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.485852 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:51.587867 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.587794 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:51.687725 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.687697 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:51.979808 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:51.979779 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:53.124722 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:53.124693 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:53.125276 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:53.124729 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:53.125276 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:53.124693 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:53.125276 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:53.124821 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:53.125276 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:53.124946 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:53.125276 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:53.125034 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:53.302500 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:53.302470 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:53.361722 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:53.361683 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:54.228544 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:54.228517 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:54.327836 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:54.327803 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:55.124980 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.124946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:55.125144 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.124946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:55.125144 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:55.125089 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ttqmm" podUID="9ed8e9d3-dce6-4a00-99aa-f300ece48f4e" Jul 08 13:23:55.125144 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.125100 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:55.125307 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:55.125203 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-np9nc" podUID="8e451a5a-2e25-4860-810e-c7d1cd7bf260" Jul 08 13:23:55.125307 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:23:55.125285 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qkjqn" podUID="a9e2da49-584e-408a-8346-adf1aae3c535" Jul 08 13:23:55.191732 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.191696 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:55.250203 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.250171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:55.346368 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.346344 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-157.ec2.internal" event="NodeReady" Jul 08 13:23:55.346522 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.346475 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 13:23:55.394163 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.394127 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2lfqh"] Jul 08 13:23:55.397568 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.397547 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zc2q9"] Jul 08 13:23:55.397687 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.397675 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.400451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.400429 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 13:23:55.401008 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.400988 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 13:23:55.401120 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.400989 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9rlw5\"" Jul 08 13:23:55.401954 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.401916 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:55.404309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.404288 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 13:23:55.404402 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.404317 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 13:23:55.404402 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.404321 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fpm48\"" Jul 08 13:23:55.404513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.404320 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 13:23:55.408845 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.408821 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zc2q9"] Jul 08 13:23:55.412091 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.412072 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2lfqh"] Jul 08 13:23:55.524939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.524891 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdkmw\" (UniqueName: \"kubernetes.io/projected/3400c925-fed7-4390-b1c3-0a84a7a57165-kube-api-access-bdkmw\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.525108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.524969 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3400c925-fed7-4390-b1c3-0a84a7a57165-tmp-dir\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.525108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.524995 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ft25\" (UniqueName: \"kubernetes.io/projected/7267733a-825c-47c8-83c4-8e9b5c044b00-kube-api-access-9ft25\") pod \"ingress-canary-zc2q9\" (UID: \"7267733a-825c-47c8-83c4-8e9b5c044b00\") " pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:55.525108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.525033 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3400c925-fed7-4390-b1c3-0a84a7a57165-metrics-tls\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.525249 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.525102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3400c925-fed7-4390-b1c3-0a84a7a57165-config-volume\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.525249 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.525145 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7267733a-825c-47c8-83c4-8e9b5c044b00-cert\") pod \"ingress-canary-zc2q9\" (UID: \"7267733a-825c-47c8-83c4-8e9b5c044b00\") " pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:55.559770 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.559740 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:55.620818 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.620742 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:55.625817 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.625796 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3400c925-fed7-4390-b1c3-0a84a7a57165-config-volume\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.625966 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.625825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7267733a-825c-47c8-83c4-8e9b5c044b00-cert\") pod \"ingress-canary-zc2q9\" (UID: \"7267733a-825c-47c8-83c4-8e9b5c044b00\") " pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:55.625966 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.625862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdkmw\" (UniqueName: \"kubernetes.io/projected/3400c925-fed7-4390-b1c3-0a84a7a57165-kube-api-access-bdkmw\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.625966 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.625902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3400c925-fed7-4390-b1c3-0a84a7a57165-tmp-dir\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.625966 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.625954 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ft25\" (UniqueName: \"kubernetes.io/projected/7267733a-825c-47c8-83c4-8e9b5c044b00-kube-api-access-9ft25\") pod \"ingress-canary-zc2q9\" (UID: \"7267733a-825c-47c8-83c4-8e9b5c044b00\") " pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:55.626174 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.625992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3400c925-fed7-4390-b1c3-0a84a7a57165-metrics-tls\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.626378 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.626358 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3400c925-fed7-4390-b1c3-0a84a7a57165-tmp-dir\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.626472 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.626451 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3400c925-fed7-4390-b1c3-0a84a7a57165-config-volume\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.630359 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.630335 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3400c925-fed7-4390-b1c3-0a84a7a57165-metrics-tls\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.630473 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.630346 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7267733a-825c-47c8-83c4-8e9b5c044b00-cert\") pod \"ingress-canary-zc2q9\" (UID: \"7267733a-825c-47c8-83c4-8e9b5c044b00\") " pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:55.633938 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.633905 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdkmw\" (UniqueName: \"kubernetes.io/projected/3400c925-fed7-4390-b1c3-0a84a7a57165-kube-api-access-bdkmw\") pod \"dns-default-2lfqh\" (UID: \"3400c925-fed7-4390-b1c3-0a84a7a57165\") " pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.634249 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.634228 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ft25\" (UniqueName: \"kubernetes.io/projected/7267733a-825c-47c8-83c4-8e9b5c044b00-kube-api-access-9ft25\") pod \"ingress-canary-zc2q9\" (UID: \"7267733a-825c-47c8-83c4-8e9b5c044b00\") " pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:55.713119 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.713072 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2lfqh" Jul 08 13:23:55.719183 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:55.719151 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zc2q9" Jul 08 13:23:57.126425 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.126075 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:23:57.127974 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.127885 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:23:57.131629 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.129739 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:23:57.131629 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.130627 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 13:23:57.134292 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.134226 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 13:23:57.134511 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.134479 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 13:23:57.134970 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.134761 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 13:23:57.135208 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.135187 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tktlz\"" Jul 08 13:23:57.135434 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.135413 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hs9z4\"" Jul 08 13:23:57.203741 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.203705 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2lfqh"] Jul 08 13:23:57.207309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.207265 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zc2q9"] Jul 08 13:23:57.285003 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:57.284893 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3400c925_fed7_4390_b1c3_0a84a7a57165.slice/crio-87d69b3dc3706fbe83c8f797fdd1a789b51991ee2bfeee9b4b1f90bed69238fe WatchSource:0}: Error finding container 87d69b3dc3706fbe83c8f797fdd1a789b51991ee2bfeee9b4b1f90bed69238fe: Status 404 returned error can't find the container with id 87d69b3dc3706fbe83c8f797fdd1a789b51991ee2bfeee9b4b1f90bed69238fe Jul 08 13:23:57.289183 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:57.289153 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7267733a_825c_47c8_83c4_8e9b5c044b00.slice/crio-3781d06b25bc3cb296debba65da9cfb6213e4bdaa9cd61517f30b5484e33194e WatchSource:0}: Error finding container 3781d06b25bc3cb296debba65da9cfb6213e4bdaa9cd61517f30b5484e33194e: Status 404 returned error can't find the container with id 3781d06b25bc3cb296debba65da9cfb6213e4bdaa9cd61517f30b5484e33194e Jul 08 13:23:57.335818 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.335776 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zc2q9" event={"ID":"7267733a-825c-47c8-83c4-8e9b5c044b00","Type":"ContainerStarted","Data":"3781d06b25bc3cb296debba65da9cfb6213e4bdaa9cd61517f30b5484e33194e"} Jul 08 13:23:57.337014 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.336987 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2lfqh" event={"ID":"3400c925-fed7-4390-b1c3-0a84a7a57165","Type":"ContainerStarted","Data":"87d69b3dc3706fbe83c8f797fdd1a789b51991ee2bfeee9b4b1f90bed69238fe"} Jul 08 13:23:57.712388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.712352 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b44fc9979-7bqzv"] Jul 08 13:23:57.725874 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.725844 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.726040 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.725946 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b44fc9979-7bqzv"] Jul 08 13:23:57.729307 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.729285 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 13:23:57.730398 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.730056 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tt79h\"" Jul 08 13:23:57.730398 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.730113 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 13:23:57.730398 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.730123 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 13:23:57.730398 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.730179 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 13:23:57.730398 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.730057 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 13:23:57.730398 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.730381 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 13:23:57.730742 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.730426 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 13:23:57.841967 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.841839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-oauth-config\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.841967 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.841890 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vctrx\" (UniqueName: \"kubernetes.io/projected/241dd353-adcc-4f84-adc1-4ad357e8cb60-kube-api-access-vctrx\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.842200 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.841986 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-serving-cert\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.842200 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.842070 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-service-ca\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.842200 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.842112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-config\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.842200 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.842138 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-oauth-serving-cert\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943054 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943012 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-config\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943054 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-oauth-serving-cert\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943274 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943122 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-oauth-config\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943274 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vctrx\" (UniqueName: \"kubernetes.io/projected/241dd353-adcc-4f84-adc1-4ad357e8cb60-kube-api-access-vctrx\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943274 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-serving-cert\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943274 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-service-ca\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943968 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943888 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-config\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.943968 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943891 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-service-ca\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.944136 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.943982 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-oauth-serving-cert\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.947468 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.947430 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-serving-cert\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.947579 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.947480 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-oauth-config\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:57.951833 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:57.951807 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vctrx\" (UniqueName: \"kubernetes.io/projected/241dd353-adcc-4f84-adc1-4ad357e8cb60-kube-api-access-vctrx\") pod \"console-b44fc9979-7bqzv\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:58.036055 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:58.036021 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:23:58.229422 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:58.229190 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b44fc9979-7bqzv"] Jul 08 13:23:58.234398 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:23:58.234359 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod241dd353_adcc_4f84_adc1_4ad357e8cb60.slice/crio-8d7be8a7413de2b56b192579b9c275f954fc3e81335b3176c468f10633949a23 WatchSource:0}: Error finding container 8d7be8a7413de2b56b192579b9c275f954fc3e81335b3176c468f10633949a23: Status 404 returned error can't find the container with id 8d7be8a7413de2b56b192579b9c275f954fc3e81335b3176c468f10633949a23 Jul 08 13:23:58.345159 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:58.345115 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9385e28-6c83-44f0-aec5-a978ae6e6192" containerID="5ab296fbd55eefc6e52f1e7a2409b4fd68bf775cbf9d3946e7fcd2b8e8634cee" exitCode=0 Jul 08 13:23:58.345472 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:58.345450 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerDied","Data":"5ab296fbd55eefc6e52f1e7a2409b4fd68bf775cbf9d3946e7fcd2b8e8634cee"} Jul 08 13:23:58.347427 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:58.347401 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44fc9979-7bqzv" event={"ID":"241dd353-adcc-4f84-adc1-4ad357e8cb60","Type":"ContainerStarted","Data":"8d7be8a7413de2b56b192579b9c275f954fc3e81335b3176c468f10633949a23"} Jul 08 13:23:59.161976 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:59.161916 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:59.353663 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:59.353632 2560 generic.go:356] "Generic (PLEG): container finished" podID="a9385e28-6c83-44f0-aec5-a978ae6e6192" containerID="ff67b0d2dbcd2c70f0fd5917f706b96fe45545df5b5c374e172a7854324009c5" exitCode=0 Jul 08 13:23:59.354195 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:59.353688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerDied","Data":"ff67b0d2dbcd2c70f0fd5917f706b96fe45545df5b5c374e172a7854324009c5"} Jul 08 13:23:59.425204 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:59.425120 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:23:59.475957 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:23:59.475904 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:01.361470 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.361438 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tz28k" event={"ID":"a9385e28-6c83-44f0-aec5-a978ae6e6192","Type":"ContainerStarted","Data":"cb00acb13205304c3d70f8edb3f7a1a8b6d24da4d5b29770bb564c4bd92ac5bb"} Jul 08 13:24:01.362898 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.362877 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zc2q9" event={"ID":"7267733a-825c-47c8-83c4-8e9b5c044b00","Type":"ContainerStarted","Data":"ecec62b14ad1bb68aa5a16f467fc1d700f01486a497ca1b5bd39c2f46f73e584"} Jul 08 13:24:01.364517 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.364488 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2lfqh" event={"ID":"3400c925-fed7-4390-b1c3-0a84a7a57165","Type":"ContainerStarted","Data":"75d65fe06db64c171cfe7beb6fd22aaf1e97dd67e07dd85f858a7c74e0c70b57"} Jul 08 13:24:01.364636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.364537 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2lfqh" event={"ID":"3400c925-fed7-4390-b1c3-0a84a7a57165","Type":"ContainerStarted","Data":"8431f7f4f8cbd5a9aac2d421c6534155bfdc046194894266fb303d00b95314aa"} Jul 08 13:24:01.364636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.364618 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2lfqh" Jul 08 13:24:01.385566 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.385528 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tz28k" podStartSLOduration=9.172836765 podStartE2EDuration="44.385515498s" totalImagesPullingTime="35.212678733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:01.385077983 +0000 UTC m=+44.830265653" watchObservedRunningTime="2026-07-08 13:24:01.385515498 +0000 UTC m=+44.830703168" Jul 08 13:24:01.401850 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.401809 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2lfqh" podStartSLOduration=3.441265646 podStartE2EDuration="6.401795516s" totalImagesPullingTime="2.96052987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:01.401149635 +0000 UTC m=+44.846337306" watchObservedRunningTime="2026-07-08 13:24:01.401795516 +0000 UTC m=+44.846983186" Jul 08 13:24:01.419985 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.419907 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zc2q9" podStartSLOduration=3.167111199 podStartE2EDuration="6.419886916s" totalImagesPullingTime="3.252775717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:01.418946908 +0000 UTC m=+44.864134578" watchObservedRunningTime="2026-07-08 13:24:01.419886916 +0000 UTC m=+44.865074587" Jul 08 13:24:01.486405 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.486354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:01.559497 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.559463 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:01.629195 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.629120 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:01.732462 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.732428 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:01.830525 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.830494 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:01.921884 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.921003 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds"] Jul 08 13:24:01.946125 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.946099 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds"] Jul 08 13:24:01.946280 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.946193 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:01.948957 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.948916 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 13:24:01.949089 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:01.949031 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-d92cn\"" Jul 08 13:24:02.020875 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.020841 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:02.073350 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.073315 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:02.075492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.075470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-562ds\" (UID: \"f83a92ef-e215-4f00-b90a-3fedcb6b45b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:02.176960 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.176861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-562ds\" (UID: \"f83a92ef-e215-4f00-b90a-3fedcb6b45b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:02.177125 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.177060 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 08 13:24:02.177177 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.177159 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates podName:f83a92ef-e215-4f00-b90a-3fedcb6b45b1 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:02.677135933 +0000 UTC m=+46.122323581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-562ds" (UID: "f83a92ef-e215-4f00-b90a-3fedcb6b45b1") : secret "prometheus-operator-admission-webhook-tls" not found Jul 08 13:24:02.193052 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.193018 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6ps5d"] Jul 08 13:24:02.212207 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.212175 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.213711 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.213684 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6ps5d"] Jul 08 13:24:02.215053 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.215016 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 13:24:02.215053 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.215040 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 13:24:02.215053 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.215046 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 13:24:02.215271 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.215099 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kvhmk\"" Jul 08 13:24:02.215990 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.215968 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 13:24:02.279555 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.279522 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:02.333226 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.333198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:02.378750 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.378717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-data-volume\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.379189 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.378761 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.379189 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.378848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.379189 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.378874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbpjm\" (UniqueName: \"kubernetes.io/projected/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-api-access-nbpjm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.379459 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.378917 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-crio-socket\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.480705 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.480632 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-data-volume\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.480705 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.480665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.480705 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.480697 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.481001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.480715 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbpjm\" (UniqueName: \"kubernetes.io/projected/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-api-access-nbpjm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.481001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.480735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-crio-socket\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.481001 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.480803 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 13:24:02.481001 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.480816 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 13:24:02.481001 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.480898 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm podName:cfb6c301-6a3c-4cc7-a065-1ec5e200a138 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:02.980878369 +0000 UTC m=+46.426066034 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ps5d" (UID: "cfb6c301-6a3c-4cc7-a065-1ec5e200a138") : configmap "kube-rbac-proxy" not found Jul 08 13:24:02.481001 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.480937 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls podName:cfb6c301-6a3c-4cc7-a065-1ec5e200a138 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:02.980911266 +0000 UTC m=+46.426098930 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ps5d" (UID: "cfb6c301-6a3c-4cc7-a065-1ec5e200a138") : secret "insights-runtime-extractor-tls" not found Jul 08 13:24:02.481001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.480992 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-crio-socket\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.481223 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.481004 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-data-volume\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.491720 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.491694 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbpjm\" (UniqueName: \"kubernetes.io/projected/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-api-access-nbpjm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.681887 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.681856 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-562ds\" (UID: \"f83a92ef-e215-4f00-b90a-3fedcb6b45b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:02.682051 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.681975 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 08 13:24:02.682051 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.682038 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates podName:f83a92ef-e215-4f00-b90a-3fedcb6b45b1 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:03.682024106 +0000 UTC m=+47.127211755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-562ds" (UID: "f83a92ef-e215-4f00-b90a-3fedcb6b45b1") : secret "prometheus-operator-admission-webhook-tls" not found Jul 08 13:24:02.883373 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.883349 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2lfqh_3400c925-fed7-4390-b1c3-0a84a7a57165/dns/0.log" Jul 08 13:24:02.896673 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.896616 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:02.984196 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.984169 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.984331 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:02.984244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:02.984331 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.984301 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 13:24:02.984417 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.984336 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 13:24:02.984417 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.984374 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm podName:cfb6c301-6a3c-4cc7-a065-1ec5e200a138 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:03.984358347 +0000 UTC m=+47.429545999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6ps5d" (UID: "cfb6c301-6a3c-4cc7-a065-1ec5e200a138") : configmap "kube-rbac-proxy" not found Jul 08 13:24:02.984417 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:02.984388 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls podName:cfb6c301-6a3c-4cc7-a065-1ec5e200a138 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:03.984382637 +0000 UTC m=+47.429570284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ps5d" (UID: "cfb6c301-6a3c-4cc7-a065-1ec5e200a138") : secret "insights-runtime-extractor-tls" not found Jul 08 13:24:03.083462 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.083435 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2lfqh_3400c925-fed7-4390-b1c3-0a84a7a57165/kube-rbac-proxy/0.log" Jul 08 13:24:03.371513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.371483 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44fc9979-7bqzv" event={"ID":"241dd353-adcc-4f84-adc1-4ad357e8cb60","Type":"ContainerStarted","Data":"d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7"} Jul 08 13:24:03.392763 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.390605 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b44fc9979-7bqzv" podStartSLOduration=1.725321209 podStartE2EDuration="6.390587491s" totalImagesPullingTime="4.665266282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:03.388701933 +0000 UTC m=+46.833889600" watchObservedRunningTime="2026-07-08 13:24:03.390587491 +0000 UTC m=+46.835775161" Jul 08 13:24:03.401295 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.401273 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:03.688936 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.688842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-562ds\" (UID: \"f83a92ef-e215-4f00-b90a-3fedcb6b45b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:03.691343 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.691314 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f83a92ef-e215-4f00-b90a-3fedcb6b45b1-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-562ds\" (UID: \"f83a92ef-e215-4f00-b90a-3fedcb6b45b1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:03.756649 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.756614 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:03.891405 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.891372 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds"] Jul 08 13:24:03.894259 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:03.894231 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83a92ef_e215_4f00_b90a_3fedcb6b45b1.slice/crio-57bb8259b93b2a3b763908d7dbcf3cdf35df41587d7bd4d7df641a1f0c6b5171 WatchSource:0}: Error finding container 57bb8259b93b2a3b763908d7dbcf3cdf35df41587d7bd4d7df641a1f0c6b5171: Status 404 returned error can't find the container with id 57bb8259b93b2a3b763908d7dbcf3cdf35df41587d7bd4d7df641a1f0c6b5171 Jul 08 13:24:03.991075 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.991002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:03.991075 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.991051 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:03.991240 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:03.991145 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 13:24:03.991240 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:03.991208 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls podName:cfb6c301-6a3c-4cc7-a065-1ec5e200a138 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:05.991191476 +0000 UTC m=+49.436379124 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6ps5d" (UID: "cfb6c301-6a3c-4cc7-a065-1ec5e200a138") : secret "insights-runtime-extractor-tls" not found Jul 08 13:24:03.991463 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:03.991448 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:04.282333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:04.282265 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v2rcg_b42a58a6-b426-475f-ba38-e40d34589fbc/dns-node-resolver/0.log" Jul 08 13:24:04.307939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:04.307897 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:04.374607 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:04.374578 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" event={"ID":"f83a92ef-e215-4f00-b90a-3fedcb6b45b1","Type":"ContainerStarted","Data":"57bb8259b93b2a3b763908d7dbcf3cdf35df41587d7bd4d7df641a1f0c6b5171"} Jul 08 13:24:04.409019 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:04.408989 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:04.703513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:04.703485 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:05.025322 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.025247 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:05.103952 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.103912 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-697fd7fc84-l76m9"] Jul 08 13:24:05.104084 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.104045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:05.118027 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.118004 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697fd7fc84-l76m9"] Jul 08 13:24:05.118165 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.118100 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.127157 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.127140 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 13:24:05.200086 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.200052 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-service-ca\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.200213 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.200090 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-oauth-serving-cert\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.200213 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.200129 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-console-config\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.200305 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.200207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-oauth-config\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.200305 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.200249 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-serving-cert\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.200305 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.200275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwvhv\" (UniqueName: \"kubernetes.io/projected/fb132285-4250-40d6-b1be-545f413ba7fa-kube-api-access-mwvhv\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.200411 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.200325 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-trusted-ca-bundle\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.283485 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.283407 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m44xj_26c9c0ad-a236-49e6-b425-7a867b9be873/node-ca/0.log" Jul 08 13:24:05.300845 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.300811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-oauth-config\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.301015 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.300858 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-serving-cert\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.301015 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.300886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwvhv\" (UniqueName: \"kubernetes.io/projected/fb132285-4250-40d6-b1be-545f413ba7fa-kube-api-access-mwvhv\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.301015 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.300950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-trusted-ca-bundle\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.301190 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.301015 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-service-ca\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.301190 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.301041 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-oauth-serving-cert\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.301190 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.301098 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-console-config\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.301988 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.301961 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-console-config\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.302549 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.302523 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-service-ca\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.303199 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.303171 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-trusted-ca-bundle\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.303686 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.303670 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-oauth-serving-cert\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.303767 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.303721 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-serving-cert\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.303959 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.303944 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-oauth-config\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.309246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.309215 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwvhv\" (UniqueName: \"kubernetes.io/projected/fb132285-4250-40d6-b1be-545f413ba7fa-kube-api-access-mwvhv\") pod \"console-697fd7fc84-l76m9\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.427431 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.427401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:05.578693 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:05.578662 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697fd7fc84-l76m9"] Jul 08 13:24:05.663108 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:05.663069 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb132285_4250_40d6_b1be_545f413ba7fa.slice/crio-d5be57b76d66243a2fc91c8e48f8951ed52ca73f469f89cc1b551643a504044f WatchSource:0}: Error finding container d5be57b76d66243a2fc91c8e48f8951ed52ca73f469f89cc1b551643a504044f: Status 404 returned error can't find the container with id d5be57b76d66243a2fc91c8e48f8951ed52ca73f469f89cc1b551643a504044f Jul 08 13:24:06.006713 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.006631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:06.009017 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.008996 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cfb6c301-6a3c-4cc7-a065-1ec5e200a138-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6ps5d\" (UID: \"cfb6c301-6a3c-4cc7-a065-1ec5e200a138\") " pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:06.124323 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.124293 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6ps5d" Jul 08 13:24:06.253246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.253217 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6ps5d"] Jul 08 13:24:06.257180 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:06.257125 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb6c301_6a3c_4cc7_a065_1ec5e200a138.slice/crio-6ce70016f8266153ee444b698618f495ecca6c11822879d2fe2b3284cb946e85 WatchSource:0}: Error finding container 6ce70016f8266153ee444b698618f495ecca6c11822879d2fe2b3284cb946e85: Status 404 returned error can't find the container with id 6ce70016f8266153ee444b698618f495ecca6c11822879d2fe2b3284cb946e85 Jul 08 13:24:06.381812 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.381776 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ps5d" event={"ID":"cfb6c301-6a3c-4cc7-a065-1ec5e200a138","Type":"ContainerStarted","Data":"6ce70016f8266153ee444b698618f495ecca6c11822879d2fe2b3284cb946e85"} Jul 08 13:24:06.383192 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.383168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd7fc84-l76m9" event={"ID":"fb132285-4250-40d6-b1be-545f413ba7fa","Type":"ContainerStarted","Data":"5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada"} Jul 08 13:24:06.383309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.383212 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd7fc84-l76m9" event={"ID":"fb132285-4250-40d6-b1be-545f413ba7fa","Type":"ContainerStarted","Data":"d5be57b76d66243a2fc91c8e48f8951ed52ca73f469f89cc1b551643a504044f"} Jul 08 13:24:06.384718 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.384693 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" event={"ID":"f83a92ef-e215-4f00-b90a-3fedcb6b45b1","Type":"ContainerStarted","Data":"4289dee91550ff5e44fad8a774fc082bee81997464cbe7845505b527fb75afe6"} Jul 08 13:24:06.384879 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.384867 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:06.389693 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.389673 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" Jul 08 13:24:06.399484 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.399444 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-697fd7fc84-l76m9" podStartSLOduration=1.399430557 podStartE2EDuration="1.399430557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:06.398548228 +0000 UTC m=+49.843735898" watchObservedRunningTime="2026-07-08 13:24:06.399430557 +0000 UTC m=+49.844618226" Jul 08 13:24:06.412050 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.412016 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-562ds" podStartSLOduration=3.638043935 podStartE2EDuration="5.412003907s" totalImagesPullingTime="1.773959972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:06.411404941 +0000 UTC m=+49.856592610" watchObservedRunningTime="2026-07-08 13:24:06.412003907 +0000 UTC m=+49.857191578" Jul 08 13:24:06.483052 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.483023 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zc2q9_7267733a-825c-47c8-83c4-8e9b5c044b00/serve-healthcheck-canary/0.log" Jul 08 13:24:06.946914 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.946878 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:06.976418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.976340 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj"] Jul 08 13:24:06.996875 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.996846 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj"] Jul 08 13:24:06.997053 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.996988 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:06.999959 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.999339 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 13:24:06.999959 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.999611 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 13:24:06.999959 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:06.999786 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 13:24:07.000542 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.000518 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 13:24:07.000801 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.000775 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bckd7\"" Jul 08 13:24:07.000955 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.000912 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 13:24:07.014574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.014547 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.014677 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.014592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.014750 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.014710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.014800 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.014780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5p6p\" (UniqueName: \"kubernetes.io/projected/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-kube-api-access-s5p6p\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.018128 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.018095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:07.115336 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.115300 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5p6p\" (UniqueName: \"kubernetes.io/projected/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-kube-api-access-s5p6p\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.115518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.115377 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.115518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.115410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.115627 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:07.115524 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 13:24:07.115627 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:07.115587 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-tls podName:075f34cd-c5b7-4c0e-ade4-e8d7aea29594 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:07.615565104 +0000 UTC m=+51.060752755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-72fnj" (UID: "075f34cd-c5b7-4c0e-ade4-e8d7aea29594") : secret "prometheus-operator-tls" not found Jul 08 13:24:07.115627 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.115621 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.116385 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.116357 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.119289 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.119247 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.130102 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.130058 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5p6p\" (UniqueName: \"kubernetes.io/projected/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-kube-api-access-s5p6p\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.359977 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.359939 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:07.464446 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.464406 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:07.566536 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.566476 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:07.620331 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.620247 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.623127 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.623094 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/075f34cd-c5b7-4c0e-ade4-e8d7aea29594-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-72fnj\" (UID: \"075f34cd-c5b7-4c0e-ade4-e8d7aea29594\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:07.909270 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:07.909183 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" Jul 08 13:24:08.036490 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:08.036450 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:24:08.036653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:08.036550 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:24:08.037549 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:08.037525 2560 patch_prober.go:28] interesting pod/console-b44fc9979-7bqzv container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.8:8443/health\": dial tcp 10.134.0.8:8443: connect: connection refused" start-of-body= Jul 08 13:24:08.037638 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:08.037589 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-b44fc9979-7bqzv" podUID="241dd353-adcc-4f84-adc1-4ad357e8cb60" containerName="console" probeResult="failure" output="Get \"https://10.134.0.8:8443/health\": dial tcp 10.134.0.8:8443: connect: connection refused" Jul 08 13:24:08.045711 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:08.045687 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj"] Jul 08 13:24:10.307224 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:10.307191 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075f34cd_c5b7_4c0e_ade4_e8d7aea29594.slice/crio-a7fbaf9e7c28fbc6fd11779272408f429b4dae255332f3ac4d1515635f100a6f WatchSource:0}: Error finding container a7fbaf9e7c28fbc6fd11779272408f429b4dae255332f3ac4d1515635f100a6f: Status 404 returned error can't find the container with id a7fbaf9e7c28fbc6fd11779272408f429b4dae255332f3ac4d1515635f100a6f Jul 08 13:24:10.401447 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:10.401410 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" event={"ID":"075f34cd-c5b7-4c0e-ade4-e8d7aea29594","Type":"ContainerStarted","Data":"a7fbaf9e7c28fbc6fd11779272408f429b4dae255332f3ac4d1515635f100a6f"} Jul 08 13:24:10.702630 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:10.702549 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:10.801837 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:10.801804 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:11.369973 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:11.369863 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2lfqh" Jul 08 13:24:11.410503 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:11.410463 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ps5d" event={"ID":"cfb6c301-6a3c-4cc7-a065-1ec5e200a138","Type":"ContainerStarted","Data":"53069f705cafb69b2658ded2266b7c27ba0cefa1ac43821160c9fe6f30c61719"} Jul 08 13:24:12.173331 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:12.173282 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:12.273476 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:12.273439 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:12.880615 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:12.880506 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:13.418943 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:13.418882 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ps5d" event={"ID":"cfb6c301-6a3c-4cc7-a065-1ec5e200a138","Type":"ContainerStarted","Data":"25183e30baecee891915cc099d13cf4fe3ea25f2df8858413ef4414df66e8adb"} Jul 08 13:24:13.420982 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:13.420955 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" event={"ID":"075f34cd-c5b7-4c0e-ade4-e8d7aea29594","Type":"ContainerStarted","Data":"fa84263e7d45a2e4cfde9d461d10ceb5d572c3c60b6553a8f383fffd6c961c26"} Jul 08 13:24:13.421116 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:13.421005 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" event={"ID":"075f34cd-c5b7-4c0e-ade4-e8d7aea29594","Type":"ContainerStarted","Data":"c927d34fb390b6471feb081f598745d44a9801cd76a168346a91de51da5d00e4"} Jul 08 13:24:13.436857 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:13.436796 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-72fnj" podStartSLOduration=5.140819362 podStartE2EDuration="7.436777112s" totalImagesPullingTime="2.29595775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:13.436377262 +0000 UTC m=+56.881564932" watchObservedRunningTime="2026-07-08 13:24:13.436777112 +0000 UTC m=+56.881964782" Jul 08 13:24:15.107124 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.107088 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:15.277687 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.277656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:15.326117 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.326081 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm"] Jul 08 13:24:15.328049 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.328033 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.331376 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.331352 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 13:24:15.331376 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.331371 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 13:24:15.331580 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.331369 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 13:24:15.331580 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.331369 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-67kjq\"" Jul 08 13:24:15.344336 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.344306 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm"] Jul 08 13:24:15.345128 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.345100 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9bg7q"] Jul 08 13:24:15.349008 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.347485 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.351702 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.351676 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 13:24:15.351702 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.351692 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 13:24:15.351877 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.351728 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dfn6w\"" Jul 08 13:24:15.351877 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.351735 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 13:24:15.364773 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.364688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:15.389821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.389786 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s92sn\" (UniqueName: \"kubernetes.io/projected/8802fd81-6a37-4b94-9231-526ae61daa5f-kube-api-access-s92sn\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390026 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.389833 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.390026 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.389871 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.390026 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.389894 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-tls\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390026 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.389911 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d009ae46-a832-47d9-927f-27117e30087c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.390026 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d009ae46-a832-47d9-927f-27117e30087c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.390269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390092 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-sys\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390121 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5pkm\" (UniqueName: \"kubernetes.io/projected/d009ae46-a832-47d9-927f-27117e30087c-kube-api-access-k5pkm\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.390269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390146 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-root\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390171 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-wtmp\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-accelerators-collector-config\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390252 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-textfile\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390471 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390301 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8802fd81-6a37-4b94-9231-526ae61daa5f-metrics-client-ca\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.390471 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.390321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.427511 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.427483 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:15.427710 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.427569 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:15.429771 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.429745 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6ps5d" event={"ID":"cfb6c301-6a3c-4cc7-a065-1ec5e200a138","Type":"ContainerStarted","Data":"82a0a947868ceaf32b4f0e60a9001f0e5abcf2dc6dac9f8059b0db47894a823a"} Jul 08 13:24:15.432356 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.432335 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:15.444793 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.444763 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:15.448096 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.448052 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6ps5d" podStartSLOduration=4.660515634 podStartE2EDuration="13.448037337s" totalImagesPullingTime="8.787521703s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:24:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:15.447572133 +0000 UTC m=+58.892759804" watchObservedRunningTime="2026-07-08 13:24:15.448037337 +0000 UTC m=+58.893225011" Jul 08 13:24:15.491535 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.491494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8802fd81-6a37-4b94-9231-526ae61daa5f-metrics-client-ca\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.491535 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.491533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.491845 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.491819 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s92sn\" (UniqueName: \"kubernetes.io/projected/8802fd81-6a37-4b94-9231-526ae61daa5f-kube-api-access-s92sn\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.491919 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.491902 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492025 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.491971 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492083 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-tls\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492144 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492120 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d009ae46-a832-47d9-927f-27117e30087c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492222 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492202 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492283 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492233 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8802fd81-6a37-4b94-9231-526ae61daa5f-metrics-client-ca\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492283 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d009ae46-a832-47d9-927f-27117e30087c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-sys\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492311 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5pkm\" (UniqueName: \"kubernetes.io/projected/d009ae46-a832-47d9-927f-27117e30087c-kube-api-access-k5pkm\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492332 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-root\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492359 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492588 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492443 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-sys\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492588 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492365 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-wtmp\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492588 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492528 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-accelerators-collector-config\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492588 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-root\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492984 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492587 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-textfile\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492984 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492641 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d009ae46-a832-47d9-927f-27117e30087c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492984 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492828 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d009ae46-a832-47d9-927f-27117e30087c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.492984 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492857 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-textfile\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.492984 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.492961 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-wtmp\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.493324 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.493294 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-accelerators-collector-config\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.494807 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.494784 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-tls\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.494910 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.494833 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.495175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.495152 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8802fd81-6a37-4b94-9231-526ae61daa5f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.495247 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.495161 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d009ae46-a832-47d9-927f-27117e30087c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.499712 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.499689 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s92sn\" (UniqueName: \"kubernetes.io/projected/8802fd81-6a37-4b94-9231-526ae61daa5f-kube-api-access-s92sn\") pod \"node-exporter-9bg7q\" (UID: \"8802fd81-6a37-4b94-9231-526ae61daa5f\") " pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.499788 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.499769 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5pkm\" (UniqueName: \"kubernetes.io/projected/d009ae46-a832-47d9-927f-27117e30087c-kube-api-access-k5pkm\") pod \"kube-state-metrics-7c9dbb8b7d-g9nbm\" (UID: \"d009ae46-a832-47d9-927f-27117e30087c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.545438 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.545397 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:15.638072 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.637989 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" Jul 08 13:24:15.643941 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.643906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:15.659995 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.659962 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9bg7q" Jul 08 13:24:15.671559 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:15.671518 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8802fd81_6a37_4b94_9231_526ae61daa5f.slice/crio-c8e5fd7824782db783999c37fd39825dea0c89d3e66297f464166fea549d39f2 WatchSource:0}: Error finding container c8e5fd7824782db783999c37fd39825dea0c89d3e66297f464166fea549d39f2: Status 404 returned error can't find the container with id c8e5fd7824782db783999c37fd39825dea0c89d3e66297f464166fea549d39f2 Jul 08 13:24:15.744819 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.744786 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:15.771099 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.771067 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm"] Jul 08 13:24:15.774536 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:15.774491 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd009ae46_a832_47d9_927f_27117e30087c.slice/crio-9ae6531f884bde19ed877e6226d4a0ba9454c8aabf1d145a92f1114a2572f363 WatchSource:0}: Error finding container 9ae6531f884bde19ed877e6226d4a0ba9454c8aabf1d145a92f1114a2572f363: Status 404 returned error can't find the container with id 9ae6531f884bde19ed877e6226d4a0ba9454c8aabf1d145a92f1114a2572f363 Jul 08 13:24:15.845418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:15.845381 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:16.325821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.325782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:16.390753 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.390720 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:16.435329 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.435279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bg7q" event={"ID":"8802fd81-6a37-4b94-9231-526ae61daa5f","Type":"ContainerStarted","Data":"c8e5fd7824782db783999c37fd39825dea0c89d3e66297f464166fea549d39f2"} Jul 08 13:24:16.437356 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.437191 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" event={"ID":"d009ae46-a832-47d9-927f-27117e30087c","Type":"ContainerStarted","Data":"9ae6531f884bde19ed877e6226d4a0ba9454c8aabf1d145a92f1114a2572f363"} Jul 08 13:24:16.442409 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.442385 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:24:16.484581 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.484544 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b44fc9979-7bqzv"] Jul 08 13:24:16.603565 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.603524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:16.906377 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:16.906296 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:17.240296 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:17.240207 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:17.338064 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:17.338018 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:17.442114 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:17.442083 2560 generic.go:356] "Generic (PLEG): container finished" podID="8802fd81-6a37-4b94-9231-526ae61daa5f" containerID="ab9568f62e4ddd28b609cffa26e1b046d18ea23ede7b69e0385dcfa533934721" exitCode=0 Jul 08 13:24:17.442244 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:17.442168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bg7q" event={"ID":"8802fd81-6a37-4b94-9231-526ae61daa5f","Type":"ContainerDied","Data":"ab9568f62e4ddd28b609cffa26e1b046d18ea23ede7b69e0385dcfa533934721"} Jul 08 13:24:18.235984 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.235949 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:18.300914 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.300879 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:18.305136 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.305110 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6cf74f759f-hmwjj"] Jul 08 13:24:18.307880 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.307851 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.311964 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.311634 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 13:24:18.314175 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.314150 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 13:24:18.314269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.314160 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 13:24:18.314269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.314240 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 13:24:18.314508 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.314436 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fvt1ssnci0grt\"" Jul 08 13:24:18.314847 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.314823 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 13:24:18.315382 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.314828 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zdl9j\"" Jul 08 13:24:18.320462 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.320398 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cf74f759f-hmwjj"] Jul 08 13:24:18.321453 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.321432 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gt94q" Jul 08 13:24:18.420464 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420426 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.420886 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420493 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.420886 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420528 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f6fd31-303a-40c8-8e37-527986051c22-metrics-client-ca\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.420886 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-tls\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.420886 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtdkx\" (UniqueName: \"kubernetes.io/projected/69f6fd31-303a-40c8-8e37-527986051c22-kube-api-access-mtdkx\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.420886 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.420886 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-grpc-tls\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.421166 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.420981 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.448332 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.448289 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bg7q" event={"ID":"8802fd81-6a37-4b94-9231-526ae61daa5f","Type":"ContainerStarted","Data":"979eddf5149143ed36fa229a6d6484fb214d53fb1ee03dfeb662337eeaf2a92c"} Jul 08 13:24:18.448531 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.448345 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9bg7q" event={"ID":"8802fd81-6a37-4b94-9231-526ae61daa5f","Type":"ContainerStarted","Data":"3900eab896d992aa7225269a9d8393b8776ce13dd425dfe6218f50b380543271"} Jul 08 13:24:18.450568 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.450545 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" event={"ID":"d009ae46-a832-47d9-927f-27117e30087c","Type":"ContainerStarted","Data":"a175e71508e84ad01e888b29d7eabf3e47dc77e59f7f24d8937c7dd007ce3ca4"} Jul 08 13:24:18.450674 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.450585 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" event={"ID":"d009ae46-a832-47d9-927f-27117e30087c","Type":"ContainerStarted","Data":"4535fc0b332d4e4f8745e380ffde3b8a8dfe0a0f4c8ad5314bd4277f58b742a7"} Jul 08 13:24:18.450674 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.450604 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" event={"ID":"d009ae46-a832-47d9-927f-27117e30087c","Type":"ContainerStarted","Data":"705eefe0b1e6a88ffe93dc2f15715f157f4d9690bce44a1c2810d7aa1bb9a10b"} Jul 08 13:24:18.459197 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.459151 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:18.467402 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.467352 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9bg7q" podStartSLOduration=2.26360339 podStartE2EDuration="3.467231791s" totalImagesPullingTime="1.116633996s" totalInitContainerRuntime="86.994405ms" isStatefulPod=true podCreationTimestamp="2026-07-08 13:24:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:18.466212718 +0000 UTC m=+61.911400388" watchObservedRunningTime="2026-07-08 13:24:18.467231791 +0000 UTC m=+61.912419441" Jul 08 13:24:18.490945 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.490811 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-g9nbm" podStartSLOduration=1.8718155300000001 podStartE2EDuration="3.490794171s" totalImagesPullingTime="1.618978641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:18.489270538 +0000 UTC m=+61.934458205" watchObservedRunningTime="2026-07-08 13:24:18.490794171 +0000 UTC m=+61.935981841" Jul 08 13:24:18.522265 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522220 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.522424 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522276 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.522424 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522323 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.522424 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f6fd31-303a-40c8-8e37-527986051c22-metrics-client-ca\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.522424 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-tls\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.523806 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtdkx\" (UniqueName: \"kubernetes.io/projected/69f6fd31-303a-40c8-8e37-527986051c22-kube-api-access-mtdkx\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.523806 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.523806 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.522953 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-grpc-tls\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.523806 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.523227 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69f6fd31-303a-40c8-8e37-527986051c22-metrics-client-ca\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.525397 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.525374 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.525744 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.525722 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.525837 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.525745 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-tls\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.525975 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.525956 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.526236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.526215 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-grpc-tls\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.526300 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.526263 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69f6fd31-303a-40c8-8e37-527986051c22-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.530848 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.530827 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtdkx\" (UniqueName: \"kubernetes.io/projected/69f6fd31-303a-40c8-8e37-527986051c22-kube-api-access-mtdkx\") pod \"thanos-querier-6cf74f759f-hmwjj\" (UID: \"69f6fd31-303a-40c8-8e37-527986051c22\") " pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.560500 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.560450 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:18.624192 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.624150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:18.659515 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.659480 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:18.764313 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:18.764288 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6cf74f759f-hmwjj"] Jul 08 13:24:18.766336 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:18.766304 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f6fd31_303a_40c8_8e37_527986051c22.slice/crio-f4030e068b63390a9814722275e5021a4733c5115bd8b4f2fd7bbc61a2616a33 WatchSource:0}: Error finding container f4030e068b63390a9814722275e5021a4733c5115bd8b4f2fd7bbc61a2616a33: Status 404 returned error can't find the container with id f4030e068b63390a9814722275e5021a4733c5115bd8b4f2fd7bbc61a2616a33 Jul 08 13:24:19.455316 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:19.455279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" event={"ID":"69f6fd31-303a-40c8-8e37-527986051c22","Type":"ContainerStarted","Data":"f4030e068b63390a9814722275e5021a4733c5115bd8b4f2fd7bbc61a2616a33"} Jul 08 13:24:20.027229 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.027191 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:20.120367 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.120337 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn"] Jul 08 13:24:20.120600 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.120387 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:20.145263 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.145230 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn"] Jul 08 13:24:20.145455 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.145375 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.147778 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.147754 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 13:24:20.148028 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.148014 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-j8p6x\"" Jul 08 13:24:20.188956 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.188902 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:20.223092 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.223057 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69689f4c44-z9q7f"] Jul 08 13:24:20.236849 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.236809 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69689f4c44-z9q7f"] Jul 08 13:24:20.237099 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.236959 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.242467 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.242438 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7f79f42-f363-4af1-9406-006f224c9ee8-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-h87vn\" (UID: \"b7f79f42-f363-4af1-9406-006f224c9ee8\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.242641 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.242577 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzlwn\" (UniqueName: \"kubernetes.io/projected/b7f79f42-f363-4af1-9406-006f224c9ee8-kube-api-access-rzlwn\") pod \"monitoring-plugin-d5dcff7cf-h87vn\" (UID: \"b7f79f42-f363-4af1-9406-006f224c9ee8\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.300476 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.300378 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:20.343403 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8r82\" (UniqueName: \"kubernetes.io/projected/39cda45a-cc07-47d1-80a9-371199017ba0-kube-api-access-x8r82\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.343584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzlwn\" (UniqueName: \"kubernetes.io/projected/b7f79f42-f363-4af1-9406-006f224c9ee8-kube-api-access-rzlwn\") pod \"monitoring-plugin-d5dcff7cf-h87vn\" (UID: \"b7f79f42-f363-4af1-9406-006f224c9ee8\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.343584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-trusted-ca-bundle\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.343584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343443 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-oauth-serving-cert\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.343584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343499 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-oauth-config\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.343584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343547 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7f79f42-f363-4af1-9406-006f224c9ee8-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-h87vn\" (UID: \"b7f79f42-f363-4af1-9406-006f224c9ee8\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.343584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343574 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-console-config\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.343812 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343647 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-serving-cert\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.343812 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:20.343668 2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 13:24:20.343812 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.343689 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-service-ca\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.343812 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:20.343734 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7f79f42-f363-4af1-9406-006f224c9ee8-monitoring-plugin-cert podName:b7f79f42-f363-4af1-9406-006f224c9ee8 nodeName:}" failed. No retries permitted until 2026-07-08 13:24:20.843719873 +0000 UTC m=+64.288907521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b7f79f42-f363-4af1-9406-006f224c9ee8-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-h87vn" (UID: "b7f79f42-f363-4af1-9406-006f224c9ee8") : secret "monitoring-plugin-cert" not found Jul 08 13:24:20.354907 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.354873 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzlwn\" (UniqueName: \"kubernetes.io/projected/b7f79f42-f363-4af1-9406-006f224c9ee8-kube-api-access-rzlwn\") pod \"monitoring-plugin-d5dcff7cf-h87vn\" (UID: \"b7f79f42-f363-4af1-9406-006f224c9ee8\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.400859 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.400823 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:20.444722 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.444682 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-serving-cert\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.444743 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-service-ca\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.444833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8r82\" (UniqueName: \"kubernetes.io/projected/39cda45a-cc07-47d1-80a9-371199017ba0-kube-api-access-x8r82\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.444869 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-trusted-ca-bundle\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.444891 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-oauth-serving-cert\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.444950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-oauth-config\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.445010 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-console-config\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445815 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.445783 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-console-config\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445948 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.445783 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-oauth-serving-cert\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.445948 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.445880 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-service-ca\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.446128 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.446085 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-trusted-ca-bundle\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.448210 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.448187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-serving-cert\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.448726 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.448709 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-oauth-config\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.453386 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.453360 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8r82\" (UniqueName: \"kubernetes.io/projected/39cda45a-cc07-47d1-80a9-371199017ba0-kube-api-access-x8r82\") pod \"console-69689f4c44-z9q7f\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.512425 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.509240 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:20.541895 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.541841 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6b45c677b7-qmx7j"] Jul 08 13:24:20.545315 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.545286 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.547735 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.547703 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 13:24:20.547898 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.547874 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 13:24:20.548272 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.547989 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-59svq\"" Jul 08 13:24:20.548272 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.547989 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 13:24:20.548272 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.547990 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 13:24:20.548272 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.548112 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 13:24:20.548829 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.548808 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:20.553343 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.553309 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 13:24:20.561326 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.561301 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b45c677b7-qmx7j"] Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.646869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.646973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-telemeter-client-tls\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.647018 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.647068 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-serving-certs-ca-bundle\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.647094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqtnz\" (UniqueName: \"kubernetes.io/projected/dca8ce09-0126-4e51-a003-4aece88b7143-kube-api-access-cqtnz\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.647168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-federate-client-tls\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.647196 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-secret-telemeter-client\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.647418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.647264 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-metrics-client-ca\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.713827 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.713796 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69689f4c44-z9q7f"] Jul 08 13:24:20.748086 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748040 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-metrics-client-ca\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748287 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748104 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748287 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-telemeter-client-tls\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748287 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748287 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748246 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-serving-certs-ca-bundle\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748287 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqtnz\" (UniqueName: \"kubernetes.io/projected/dca8ce09-0126-4e51-a003-4aece88b7143-kube-api-access-cqtnz\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748553 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748314 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-federate-client-tls\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748553 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-secret-telemeter-client\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.748988 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.748953 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-metrics-client-ca\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.749752 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.749731 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-serving-certs-ca-bundle\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.750567 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.750543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dca8ce09-0126-4e51-a003-4aece88b7143-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.750941 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.750900 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.751449 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.751425 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-secret-telemeter-client\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.751552 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.751507 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-telemeter-client-tls\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.751612 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.751547 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dca8ce09-0126-4e51-a003-4aece88b7143-federate-client-tls\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.756985 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.756962 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqtnz\" (UniqueName: \"kubernetes.io/projected/dca8ce09-0126-4e51-a003-4aece88b7143-kube-api-access-cqtnz\") pod \"telemeter-client-6b45c677b7-qmx7j\" (UID: \"dca8ce09-0126-4e51-a003-4aece88b7143\") " pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:20.849022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.848980 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7f79f42-f363-4af1-9406-006f224c9ee8-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-h87vn\" (UID: \"b7f79f42-f363-4af1-9406-006f224c9ee8\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.851734 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.851704 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7f79f42-f363-4af1-9406-006f224c9ee8-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-h87vn\" (UID: \"b7f79f42-f363-4af1-9406-006f224c9ee8\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:20.857802 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:20.857769 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" Jul 08 13:24:21.056712 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.056674 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:21.212782 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:21.212702 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cda45a_cc07_47d1_80a9_371199017ba0.slice/crio-fd157c85c33ffbbdd1404cdf33de6a5a84dec23a910231c13ac66b7601603579 WatchSource:0}: Error finding container fd157c85c33ffbbdd1404cdf33de6a5a84dec23a910231c13ac66b7601603579: Status 404 returned error can't find the container with id fd157c85c33ffbbdd1404cdf33de6a5a84dec23a910231c13ac66b7601603579 Jul 08 13:24:21.216182 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.216147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:21.329370 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.329241 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:21.404883 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.404846 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn"] Jul 08 13:24:21.407734 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.407674 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:21.469962 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.469868 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" event={"ID":"69f6fd31-303a-40c8-8e37-527986051c22","Type":"ContainerStarted","Data":"d5a0e69a4cdea6c55f29eb737e4230ba6f0431aba82b3ed0610eee2496b22253"} Jul 08 13:24:21.470102 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.470032 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" event={"ID":"69f6fd31-303a-40c8-8e37-527986051c22","Type":"ContainerStarted","Data":"933d9a32836462cd0a5e3a30ff4c1e3304877f02c0feb3b3cfe979b5b670f56e"} Jul 08 13:24:21.471586 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.471556 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69689f4c44-z9q7f" event={"ID":"39cda45a-cc07-47d1-80a9-371199017ba0","Type":"ContainerStarted","Data":"38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7"} Jul 08 13:24:21.472214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.472190 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69689f4c44-z9q7f" event={"ID":"39cda45a-cc07-47d1-80a9-371199017ba0","Type":"ContainerStarted","Data":"fd157c85c33ffbbdd1404cdf33de6a5a84dec23a910231c13ac66b7601603579"} Jul 08 13:24:21.473184 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.473164 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" event={"ID":"b7f79f42-f363-4af1-9406-006f224c9ee8","Type":"ContainerStarted","Data":"de16f5db4e1a970d48bff25f6ef734259791d757fa0b3212d522e679872b84a4"} Jul 08 13:24:21.491617 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.491556 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69689f4c44-z9q7f" podStartSLOduration=1.49153918 podStartE2EDuration="1.49153918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:21.490747934 +0000 UTC m=+64.935935605" watchObservedRunningTime="2026-07-08 13:24:21.49153918 +0000 UTC m=+64.936726850" Jul 08 13:24:21.589378 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.589342 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:21.590647 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.590623 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 13:24:21.593799 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.593778 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.597363 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.597101 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 13:24:21.597363 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.597232 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 13:24:21.597363 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.597246 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 13:24:21.597363 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.597289 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 13:24:21.597363 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.597106 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 13:24:21.597734 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.597574 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 13:24:21.597852 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.597809 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 13:24:21.598344 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598318 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 13:24:21.598486 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598405 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bpwjq\"" Jul 08 13:24:21.598576 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598518 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 13:24:21.598646 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598617 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 13:24:21.598746 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598725 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 13:24:21.598857 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598802 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 13:24:21.598857 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598821 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 13:24:21.598995 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.598978 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a82o94inoukqm\"" Jul 08 13:24:21.600573 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.600552 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 13:24:21.603965 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.603907 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 13:24:21.616320 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.616284 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 13:24:21.618536 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.618488 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b45c677b7-qmx7j"] Jul 08 13:24:21.624601 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:21.624555 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca8ce09_0126_4e51_a003_4aece88b7143.slice/crio-7c8131d8c26ac7406d109263cd5bb3d4fd1a3a3686b190706dfa790c72cdd5c3 WatchSource:0}: Error finding container 7c8131d8c26ac7406d109263cd5bb3d4fd1a3a3686b190706dfa790c72cdd5c3: Status 404 returned error can't find the container with id 7c8131d8c26ac7406d109263cd5bb3d4fd1a3a3686b190706dfa790c72cdd5c3 Jul 08 13:24:21.658704 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658667 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658704 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658731 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658750 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658775 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658811 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658889 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1384ded-9af6-4b93-840c-14ceff895960-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658907 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.658945 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw6rn\" (UniqueName: \"kubernetes.io/projected/f1384ded-9af6-4b93-840c-14ceff895960-kube-api-access-gw6rn\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.658999 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659001 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-web-config\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659040 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659123 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1384ded-9af6-4b93-840c-14ceff895960-config-out\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659155 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659175 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659191 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659250 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659272 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-config\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659304 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.659380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.659350 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761016 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.760884 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-web-config\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761016 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.760985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761026 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1384ded-9af6-4b93-840c-14ceff895960-config-out\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761051 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761092 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761134 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-config\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761271 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761307 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761405 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761459 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761510 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1384ded-9af6-4b93-840c-14ceff895960-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761535 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.761574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.761566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw6rn\" (UniqueName: \"kubernetes.io/projected/f1384ded-9af6-4b93-840c-14ceff895960-kube-api-access-gw6rn\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.762614 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.762578 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.763624 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.763297 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.763742 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.763656 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.764450 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.764384 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.764677 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.764641 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.765178 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.765144 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.766200 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.766169 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.766587 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.766544 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f1384ded-9af6-4b93-840c-14ceff895960-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.767428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.767357 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.767626 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.767499 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.768519 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.768496 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f1384ded-9af6-4b93-840c-14ceff895960-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.768793 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.768749 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.769067 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.769047 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.769182 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.769076 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.769280 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.769257 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.769425 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.769409 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-web-config\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.770131 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.770112 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-config\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.770402 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.770380 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw6rn\" (UniqueName: \"kubernetes.io/projected/f1384ded-9af6-4b93-840c-14ceff895960-kube-api-access-gw6rn\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.770632 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.770617 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f1384ded-9af6-4b93-840c-14ceff895960-config-out\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.770712 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.770635 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f1384ded-9af6-4b93-840c-14ceff895960-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f1384ded-9af6-4b93-840c-14ceff895960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.862728 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.862688 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:24:21.862960 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.862737 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:24:21.865736 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.865317 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 13:24:21.865736 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.865606 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 13:24:21.876337 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.876242 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9ed8e9d3-dce6-4a00-99aa-f300ece48f4e-original-pull-secret\") pod \"global-pull-secret-syncer-ttqmm\" (UID: \"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e\") " pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:24:21.876337 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.876254 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8e451a5a-2e25-4860-810e-c7d1cd7bf260-metrics-certs\") pod \"network-metrics-daemon-np9nc\" (UID: \"8e451a5a-2e25-4860-810e-c7d1cd7bf260\") " pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:24:21.906600 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.906563 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:21.964760 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.964703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:24:21.967646 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.967621 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 13:24:21.978758 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.978729 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 13:24:21.988847 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:21.988816 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hbxq\" (UniqueName: \"kubernetes.io/projected/a9e2da49-584e-408a-8346-adf1aae3c535-kube-api-access-9hbxq\") pod \"network-check-target-qkjqn\" (UID: \"a9e2da49-584e-408a-8346-adf1aae3c535\") " pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:24:22.045709 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.045614 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ttqmm" Jul 08 13:24:22.055692 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.055655 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tktlz\"" Jul 08 13:24:22.063457 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.063428 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-np9nc" Jul 08 13:24:22.063710 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.063692 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hs9z4\"" Jul 08 13:24:22.072338 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.072315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:24:22.080647 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.080615 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 13:24:22.083865 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:22.083835 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1384ded_9af6_4b93_840c_14ceff895960.slice/crio-40c6dbb18fbe37a3faa4dc7610135e712cdbec7e77c4543c0c44a54f29f4b887 WatchSource:0}: Error finding container 40c6dbb18fbe37a3faa4dc7610135e712cdbec7e77c4543c0c44a54f29f4b887: Status 404 returned error can't find the container with id 40c6dbb18fbe37a3faa4dc7610135e712cdbec7e77c4543c0c44a54f29f4b887 Jul 08 13:24:22.196533 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.196392 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:22.223810 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.223100 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ttqmm"] Jul 08 13:24:22.231082 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:22.230970 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed8e9d3_dce6_4a00_99aa_f300ece48f4e.slice/crio-27a20640ab8f74364f905a98efad4a2b7bfc7629a7b864856a2ca8df13898cc5 WatchSource:0}: Error finding container 27a20640ab8f74364f905a98efad4a2b7bfc7629a7b864856a2ca8df13898cc5: Status 404 returned error can't find the container with id 27a20640ab8f74364f905a98efad4a2b7bfc7629a7b864856a2ca8df13898cc5 Jul 08 13:24:22.244728 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.244686 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-np9nc"] Jul 08 13:24:22.248086 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.248045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:22.276180 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.276139 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qkjqn"] Jul 08 13:24:22.280190 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:22.280154 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e2da49_584e_408a_8346_adf1aae3c535.slice/crio-4dc24c8710b72bce43a2b3b061df4a1a00593febeff5852e9f47a057ce7cb905 WatchSource:0}: Error finding container 4dc24c8710b72bce43a2b3b061df4a1a00593febeff5852e9f47a057ce7cb905: Status 404 returned error can't find the container with id 4dc24c8710b72bce43a2b3b061df4a1a00593febeff5852e9f47a057ce7cb905 Jul 08 13:24:22.479959 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.479904 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qkjqn" event={"ID":"a9e2da49-584e-408a-8346-adf1aae3c535","Type":"ContainerStarted","Data":"4dc24c8710b72bce43a2b3b061df4a1a00593febeff5852e9f47a057ce7cb905"} Jul 08 13:24:22.481645 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.481608 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerStarted","Data":"40c6dbb18fbe37a3faa4dc7610135e712cdbec7e77c4543c0c44a54f29f4b887"} Jul 08 13:24:22.483173 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.483148 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" event={"ID":"dca8ce09-0126-4e51-a003-4aece88b7143","Type":"ContainerStarted","Data":"7c8131d8c26ac7406d109263cd5bb3d4fd1a3a3686b190706dfa790c72cdd5c3"} Jul 08 13:24:22.484538 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.484497 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-np9nc" event={"ID":"8e451a5a-2e25-4860-810e-c7d1cd7bf260","Type":"ContainerStarted","Data":"ce4efb4b250d6fbc9887dd9a10dca6d72119613299217bbd9809ab0d51ed1fb1"} Jul 08 13:24:22.485768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.485738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ttqmm" event={"ID":"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e","Type":"ContainerStarted","Data":"27a20640ab8f74364f905a98efad4a2b7bfc7629a7b864856a2ca8df13898cc5"} Jul 08 13:24:22.489701 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:22.489667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" event={"ID":"69f6fd31-303a-40c8-8e37-527986051c22","Type":"ContainerStarted","Data":"d766f6b36cb75eb3a9b2a0cd5241ae1cfc35c6f15e89be2febf4c05ea370bb80"} Jul 08 13:24:23.499768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:23.499037 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" event={"ID":"69f6fd31-303a-40c8-8e37-527986051c22","Type":"ContainerStarted","Data":"d9cde572eba5d97b12e6a57acd2326cab2932b3fb34f0fced25449e40a1a2c4a"} Jul 08 13:24:23.499768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:23.499111 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" event={"ID":"69f6fd31-303a-40c8-8e37-527986051c22","Type":"ContainerStarted","Data":"c6cb83250164a31768ffc3a2b6bf2e20ca35f3202179a0eb8ef6db4170ca1138"} Jul 08 13:24:23.499768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:23.499152 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" event={"ID":"69f6fd31-303a-40c8-8e37-527986051c22","Type":"ContainerStarted","Data":"c96981c3e687e1af51799db5cf2789405ea57c18125c705d430c8bbe6586799d"} Jul 08 13:24:23.499768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:23.499683 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:23.528075 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:23.527545 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" podStartSLOduration=2.076414886 podStartE2EDuration="5.527520458s" totalImagesPullingTime="3.451105572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:23.524357685 +0000 UTC m=+66.969545355" watchObservedRunningTime="2026-07-08 13:24:23.527520458 +0000 UTC m=+66.972708132" Jul 08 13:24:24.867974 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:24.867905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:24.928988 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:24.928946 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:26.191766 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:26.191640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:26.310715 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:26.310639 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:28.527742 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.527699 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-np9nc" event={"ID":"8e451a5a-2e25-4860-810e-c7d1cd7bf260","Type":"ContainerStarted","Data":"320b516a9416ce5369d6da0cfd1cf740e73edb1c8aaf8a891acad814dfa7226a"} Jul 08 13:24:28.528171 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.527790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-np9nc" event={"ID":"8e451a5a-2e25-4860-810e-c7d1cd7bf260","Type":"ContainerStarted","Data":"78518fa24a363e5aca84281d8219840feb0bbdbc6e39e620e383d2e00591bd88"} Jul 08 13:24:28.529248 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.529225 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ttqmm" event={"ID":"9ed8e9d3-dce6-4a00-99aa-f300ece48f4e","Type":"ContainerStarted","Data":"ffc3ab28019b6bdfb1823294be333f6e3cce8ed5d74b428517810871a4876a6e"} Jul 08 13:24:28.530841 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.530817 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qkjqn" event={"ID":"a9e2da49-584e-408a-8346-adf1aae3c535","Type":"ContainerStarted","Data":"a0b1dacdedc223d8cabfd7e6ce8a480f77d6108895a0922de4c39e154a08e7fc"} Jul 08 13:24:28.531008 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.530875 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:24:28.532650 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.532630 2560 generic.go:356] "Generic (PLEG): container finished" podID="f1384ded-9af6-4b93-840c-14ceff895960" containerID="c02fa9b54577c94af67c89c5b06cab82214051f5e9a575b50cb6b264a7bf1b3b" exitCode=0 Jul 08 13:24:28.532787 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.532707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerDied","Data":"c02fa9b54577c94af67c89c5b06cab82214051f5e9a575b50cb6b264a7bf1b3b"} Jul 08 13:24:28.534870 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.534850 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" event={"ID":"dca8ce09-0126-4e51-a003-4aece88b7143","Type":"ContainerStarted","Data":"6a14ee516d4fc085df5bdd354764ab2b2f676fed49b58a0c415c9af60f6a01ac"} Jul 08 13:24:28.534960 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.534909 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" event={"ID":"dca8ce09-0126-4e51-a003-4aece88b7143","Type":"ContainerStarted","Data":"36b3d2284c3b5458501bca9702c4c53a761d86a62436a2061536f1303eaf927b"} Jul 08 13:24:28.535010 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.534963 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" event={"ID":"dca8ce09-0126-4e51-a003-4aece88b7143","Type":"ContainerStarted","Data":"fe251338b677350ba9410dc1837913a4eaf62f82a31f9eef9e3e0eabdedf3cb2"} Jul 08 13:24:28.536324 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.536307 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" event={"ID":"b7f79f42-f363-4af1-9406-006f224c9ee8","Type":"ContainerStarted","Data":"6612d6239e6e64cd086d79f13b5ad53a8168bbc2704d7eca2c75b0ce5ae3fa4e"} Jul 08 13:24:28.536515 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.536501 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:28.541231 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.541212 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" Jul 08 13:24:28.545480 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.545427 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-np9nc" podStartSLOduration=66.350093383 podStartE2EDuration="1m11.545415902s" totalImagesPullingTime="5.195322519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:28.54472902 +0000 UTC m=+71.989916690" watchObservedRunningTime="2026-07-08 13:24:28.545415902 +0000 UTC m=+71.990603571" Jul 08 13:24:28.571269 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.571208 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6b45c677b7-qmx7j" podStartSLOduration=2.753065 podStartE2EDuration="8.571190434s" totalImagesPullingTime="5.818125434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:28.569345474 +0000 UTC m=+72.014533134" watchObservedRunningTime="2026-07-08 13:24:28.571190434 +0000 UTC m=+72.016378100" Jul 08 13:24:28.590694 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.590639 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-h87vn" podStartSLOduration=2.565169899 podStartE2EDuration="8.59061939s" totalImagesPullingTime="6.025449491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:28.588833459 +0000 UTC m=+72.034021128" watchObservedRunningTime="2026-07-08 13:24:28.59061939 +0000 UTC m=+72.035807059" Jul 08 13:24:28.609400 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.609345 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qkjqn" podStartSLOduration=65.432723445 podStartE2EDuration="1m10.609327665s" totalImagesPullingTime="5.17660422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:28.607316606 +0000 UTC m=+72.052504275" watchObservedRunningTime="2026-07-08 13:24:28.609327665 +0000 UTC m=+72.054515335" Jul 08 13:24:28.630043 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.629985 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ttqmm" podStartSLOduration=65.398289539 podStartE2EDuration="1m10.629966179s" totalImagesPullingTime="5.23167664s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 13:23:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:28.626872117 +0000 UTC m=+72.072059800" watchObservedRunningTime="2026-07-08 13:24:28.629966179 +0000 UTC m=+72.075153858" Jul 08 13:24:28.678939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.678893 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:28.776216 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:28.776128 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:29.313221 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.313184 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69689f4c44-z9q7f"] Jul 08 13:24:29.346237 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.346204 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c96cc7f4b-pk2mk"] Jul 08 13:24:29.350535 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.350515 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.358503 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.358476 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c96cc7f4b-pk2mk"] Jul 08 13:24:29.434086 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.434039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-oauth-config\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.434277 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.434108 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-service-ca\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.434277 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.434132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-trusted-ca-bundle\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.434277 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.434159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-oauth-serving-cert\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.434399 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.434280 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-serving-cert\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.434399 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.434330 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh7h9\" (UniqueName: \"kubernetes.io/projected/11a4777a-0ee8-4994-9116-bc566e1ea3f9-kube-api-access-zh7h9\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.434399 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.434372 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-config\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.513066 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.513029 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6cf74f759f-hmwjj" Jul 08 13:24:29.536374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.535802 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-serving-cert\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.536374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.536092 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh7h9\" (UniqueName: \"kubernetes.io/projected/11a4777a-0ee8-4994-9116-bc566e1ea3f9-kube-api-access-zh7h9\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.536374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.536249 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-config\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.536374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.536299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-oauth-config\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.536374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.536340 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-service-ca\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.539493 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.538993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-trusted-ca-bundle\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.539493 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.539050 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-oauth-serving-cert\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.541094 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.541035 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-service-ca\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.541094 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.541034 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-oauth-serving-cert\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.541272 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.541127 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-trusted-ca-bundle\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.541465 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.541436 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-config\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.543753 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.543723 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-serving-cert\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.549689 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.549632 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh7h9\" (UniqueName: \"kubernetes.io/projected/11a4777a-0ee8-4994-9116-bc566e1ea3f9-kube-api-access-zh7h9\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.554792 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.553624 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-oauth-config\") pod \"console-7c96cc7f4b-pk2mk\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.660971 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.660847 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:29.838915 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:29.838869 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c96cc7f4b-pk2mk"] Jul 08 13:24:29.844088 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:24:29.844049 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a4777a_0ee8_4994_9116_bc566e1ea3f9.slice/crio-9945584b2fcf78f34ae8d37d41405461043ad5508b49211efcd3d3bfe4d7bb71 WatchSource:0}: Error finding container 9945584b2fcf78f34ae8d37d41405461043ad5508b49211efcd3d3bfe4d7bb71: Status 404 returned error can't find the container with id 9945584b2fcf78f34ae8d37d41405461043ad5508b49211efcd3d3bfe4d7bb71 Jul 08 13:24:30.549311 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:30.549270 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:30.555305 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:30.555198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c96cc7f4b-pk2mk" event={"ID":"11a4777a-0ee8-4994-9116-bc566e1ea3f9","Type":"ContainerStarted","Data":"af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679"} Jul 08 13:24:30.555305 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:30.555264 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c96cc7f4b-pk2mk" event={"ID":"11a4777a-0ee8-4994-9116-bc566e1ea3f9","Type":"ContainerStarted","Data":"9945584b2fcf78f34ae8d37d41405461043ad5508b49211efcd3d3bfe4d7bb71"} Jul 08 13:24:30.573408 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:30.573321 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c96cc7f4b-pk2mk" podStartSLOduration=1.5732978549999999 podStartE2EDuration="1.573297855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:30.571991501 +0000 UTC m=+74.017179171" watchObservedRunningTime="2026-07-08 13:24:30.573297855 +0000 UTC m=+74.018485528" Jul 08 13:24:33.571280 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:33.571251 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerStarted","Data":"adaafcdc7eedb0becf97e345e423212681d6f43578c8498cf221b716630d01a2"} Jul 08 13:24:33.985346 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:33.985315 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:34.085417 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:34.085368 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:34.578834 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:34.578791 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerStarted","Data":"4ff074cbd322eb62aaf3b6aa77c2982b022e89f05a5341362d08d3dfb608abe5"} Jul 08 13:24:34.579270 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:34.578887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerStarted","Data":"08cb4f1ece9ab2e9a938374d21d5026fab8b6478c2ccb9738842763bf5c643e7"} Jul 08 13:24:34.579270 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:34.578943 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerStarted","Data":"83e8bec1f2cbda73650a616119c72ab92710d9ccf939e42cdeef07035d0b2ebd"} Jul 08 13:24:34.579270 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:34.578971 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerStarted","Data":"eb5f65a3023e4981a98567cd22bb49677f5b66444a8d0bce63f7bfc601f1453b"} Jul 08 13:24:34.579270 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:34.578995 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f1384ded-9af6-4b93-840c-14ceff895960","Type":"ContainerStarted","Data":"d8bfaea63d987f4f316137ff176bab554e86c15917a6066182cba1f9f7bb7d10"} Jul 08 13:24:34.608306 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:34.607376 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.391362805 podStartE2EDuration="13.607345792s" totalImagesPullingTime="10.182522072s" totalInitContainerRuntime="33.460915ms" isStatefulPod=false podCreationTimestamp="2026-07-08 13:24:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:24:34.604787914 +0000 UTC m=+78.049975586" watchObservedRunningTime="2026-07-08 13:24:34.607345792 +0000 UTC m=+78.052533463" Jul 08 13:24:35.063369 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:35.063332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:35.116592 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:35.116560 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:36.907216 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:36.907175 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:24:39.661318 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:39.661187 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:39.661318 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:39.661267 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:39.666059 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:39.666035 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:40.609202 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:40.609173 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:24:40.658502 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:40.658456 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697fd7fc84-l76m9"] Jul 08 13:24:40.759002 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:40.758960 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:40.812112 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:40.812060 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:40.873276 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:40.873151 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:41.511312 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.511244 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-b44fc9979-7bqzv" podUID="241dd353-adcc-4f84-adc1-4ad357e8cb60" containerName="console" containerID="cri-o://d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7" gracePeriod=15 Jul 08 13:24:41.753298 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.753274 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:24:41.852903 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.852860 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-service-ca\") pod \"241dd353-adcc-4f84-adc1-4ad357e8cb60\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " Jul 08 13:24:41.853366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.852957 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vctrx\" (UniqueName: \"kubernetes.io/projected/241dd353-adcc-4f84-adc1-4ad357e8cb60-kube-api-access-vctrx\") pod \"241dd353-adcc-4f84-adc1-4ad357e8cb60\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " Jul 08 13:24:41.853366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.852990 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-oauth-config\") pod \"241dd353-adcc-4f84-adc1-4ad357e8cb60\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " Jul 08 13:24:41.853366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.853045 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-config\") pod \"241dd353-adcc-4f84-adc1-4ad357e8cb60\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " Jul 08 13:24:41.853366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.853071 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-serving-cert\") pod \"241dd353-adcc-4f84-adc1-4ad357e8cb60\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " Jul 08 13:24:41.853366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.853089 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-oauth-serving-cert\") pod \"241dd353-adcc-4f84-adc1-4ad357e8cb60\" (UID: \"241dd353-adcc-4f84-adc1-4ad357e8cb60\") " Jul 08 13:24:41.853633 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.853389 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-service-ca" (OuterVolumeSpecName: "service-ca") pod "241dd353-adcc-4f84-adc1-4ad357e8cb60" (UID: "241dd353-adcc-4f84-adc1-4ad357e8cb60"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:41.853633 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.853562 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-config" (OuterVolumeSpecName: "console-config") pod "241dd353-adcc-4f84-adc1-4ad357e8cb60" (UID: "241dd353-adcc-4f84-adc1-4ad357e8cb60"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:41.853633 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.853597 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "241dd353-adcc-4f84-adc1-4ad357e8cb60" (UID: "241dd353-adcc-4f84-adc1-4ad357e8cb60"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:41.855483 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.855456 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "241dd353-adcc-4f84-adc1-4ad357e8cb60" (UID: "241dd353-adcc-4f84-adc1-4ad357e8cb60"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:24:41.855588 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.855505 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/241dd353-adcc-4f84-adc1-4ad357e8cb60-kube-api-access-vctrx" (OuterVolumeSpecName: "kube-api-access-vctrx") pod "241dd353-adcc-4f84-adc1-4ad357e8cb60" (UID: "241dd353-adcc-4f84-adc1-4ad357e8cb60"). InnerVolumeSpecName "kube-api-access-vctrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:24:41.855588 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.855521 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "241dd353-adcc-4f84-adc1-4ad357e8cb60" (UID: "241dd353-adcc-4f84-adc1-4ad357e8cb60"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:24:41.954246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.954191 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vctrx\" (UniqueName: \"kubernetes.io/projected/241dd353-adcc-4f84-adc1-4ad357e8cb60-kube-api-access-vctrx\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:41.954246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.954237 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-oauth-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:41.954246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.954250 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:41.954246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.954258 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/241dd353-adcc-4f84-adc1-4ad357e8cb60-console-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:41.954246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.954269 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-oauth-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:41.954549 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:41.954277 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/241dd353-adcc-4f84-adc1-4ad357e8cb60-service-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:42.615768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.615730 2560 generic.go:356] "Generic (PLEG): container finished" podID="241dd353-adcc-4f84-adc1-4ad357e8cb60" containerID="d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7" exitCode=0 Jul 08 13:24:42.615963 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.615774 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b44fc9979-7bqzv" Jul 08 13:24:42.615963 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.615824 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44fc9979-7bqzv" event={"ID":"241dd353-adcc-4f84-adc1-4ad357e8cb60","Type":"ContainerDied","Data":"d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7"} Jul 08 13:24:42.615963 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.615872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b44fc9979-7bqzv" event={"ID":"241dd353-adcc-4f84-adc1-4ad357e8cb60","Type":"ContainerDied","Data":"8d7be8a7413de2b56b192579b9c275f954fc3e81335b3176c468f10633949a23"} Jul 08 13:24:42.615963 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.615892 2560 scope.go:117] "RemoveContainer" containerID="d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7" Jul 08 13:24:42.626584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.626550 2560 scope.go:117] "RemoveContainer" containerID="d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7" Jul 08 13:24:42.626943 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:42.626905 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7\": container with ID starting with d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7 not found: ID does not exist" containerID="d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7" Jul 08 13:24:42.627006 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.626960 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7"} err="failed to get container status \"d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7\": rpc error: code = NotFound desc = could not find container \"d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7\": container with ID starting with d543fc1104c4aba7400fab569bd8ee907454799d27725f0bb3b61f4722c7deb7 not found: ID does not exist" Jul 08 13:24:42.636014 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.635982 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b44fc9979-7bqzv"] Jul 08 13:24:42.639751 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:42.639721 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b44fc9979-7bqzv"] Jul 08 13:24:43.129501 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:43.129466 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="241dd353-adcc-4f84-adc1-4ad357e8cb60" path="/var/lib/kubelet/pods/241dd353-adcc-4f84-adc1-4ad357e8cb60/volumes" Jul 08 13:24:48.248636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:48.248604 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:50.024918 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:50.024883 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:50.125416 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:50.125385 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:54.333513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.333467 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-69689f4c44-z9q7f" podUID="39cda45a-cc07-47d1-80a9-371199017ba0" containerName="console" containerID="cri-o://38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7" gracePeriod=15 Jul 08 13:24:54.572700 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.572679 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:54.654566 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654494 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8r82\" (UniqueName: \"kubernetes.io/projected/39cda45a-cc07-47d1-80a9-371199017ba0-kube-api-access-x8r82\") pod \"39cda45a-cc07-47d1-80a9-371199017ba0\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " Jul 08 13:24:54.654566 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654529 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-trusted-ca-bundle\") pod \"39cda45a-cc07-47d1-80a9-371199017ba0\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " Jul 08 13:24:54.654566 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654549 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-serving-cert\") pod \"39cda45a-cc07-47d1-80a9-371199017ba0\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " Jul 08 13:24:54.654765 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654681 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-service-ca\") pod \"39cda45a-cc07-47d1-80a9-371199017ba0\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " Jul 08 13:24:54.654765 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654747 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-console-config\") pod \"39cda45a-cc07-47d1-80a9-371199017ba0\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " Jul 08 13:24:54.654841 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654778 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-oauth-serving-cert\") pod \"39cda45a-cc07-47d1-80a9-371199017ba0\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " Jul 08 13:24:54.654841 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654808 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-oauth-config\") pod \"39cda45a-cc07-47d1-80a9-371199017ba0\" (UID: \"39cda45a-cc07-47d1-80a9-371199017ba0\") " Jul 08 13:24:54.655141 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.655106 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-console-config" (OuterVolumeSpecName: "console-config") pod "39cda45a-cc07-47d1-80a9-371199017ba0" (UID: "39cda45a-cc07-47d1-80a9-371199017ba0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:54.655236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.655132 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-service-ca" (OuterVolumeSpecName: "service-ca") pod "39cda45a-cc07-47d1-80a9-371199017ba0" (UID: "39cda45a-cc07-47d1-80a9-371199017ba0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:54.655236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.655192 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "39cda45a-cc07-47d1-80a9-371199017ba0" (UID: "39cda45a-cc07-47d1-80a9-371199017ba0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:54.655485 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.654915 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "39cda45a-cc07-47d1-80a9-371199017ba0" (UID: "39cda45a-cc07-47d1-80a9-371199017ba0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:24:54.656947 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.656909 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39cda45a-cc07-47d1-80a9-371199017ba0-kube-api-access-x8r82" (OuterVolumeSpecName: "kube-api-access-x8r82") pod "39cda45a-cc07-47d1-80a9-371199017ba0" (UID: "39cda45a-cc07-47d1-80a9-371199017ba0"). InnerVolumeSpecName "kube-api-access-x8r82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:24:54.657014 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.656965 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "39cda45a-cc07-47d1-80a9-371199017ba0" (UID: "39cda45a-cc07-47d1-80a9-371199017ba0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:24:54.657225 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.657203 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "39cda45a-cc07-47d1-80a9-371199017ba0" (UID: "39cda45a-cc07-47d1-80a9-371199017ba0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:24:54.669939 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.669901 2560 generic.go:356] "Generic (PLEG): container finished" podID="39cda45a-cc07-47d1-80a9-371199017ba0" containerID="38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7" exitCode=0 Jul 08 13:24:54.670036 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.669957 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69689f4c44-z9q7f" event={"ID":"39cda45a-cc07-47d1-80a9-371199017ba0","Type":"ContainerDied","Data":"38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7"} Jul 08 13:24:54.670036 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.670006 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69689f4c44-z9q7f" event={"ID":"39cda45a-cc07-47d1-80a9-371199017ba0","Type":"ContainerDied","Data":"fd157c85c33ffbbdd1404cdf33de6a5a84dec23a910231c13ac66b7601603579"} Jul 08 13:24:54.670036 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.669968 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69689f4c44-z9q7f" Jul 08 13:24:54.670036 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.670027 2560 scope.go:117] "RemoveContainer" containerID="38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7" Jul 08 13:24:54.678660 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.678643 2560 scope.go:117] "RemoveContainer" containerID="38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7" Jul 08 13:24:54.678888 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:24:54.678870 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7\": container with ID starting with 38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7 not found: ID does not exist" containerID="38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7" Jul 08 13:24:54.678956 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.678903 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7"} err="failed to get container status \"38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7\": rpc error: code = NotFound desc = could not find container \"38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7\": container with ID starting with 38d987c7f10815f52c8f32fbc93c50147945a4089c2c883c865c584aa35326d7 not found: ID does not exist" Jul 08 13:24:54.691033 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.691008 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69689f4c44-z9q7f"] Jul 08 13:24:54.694368 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.694350 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69689f4c44-z9q7f"] Jul 08 13:24:54.756460 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.756438 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8r82\" (UniqueName: \"kubernetes.io/projected/39cda45a-cc07-47d1-80a9-371199017ba0-kube-api-access-x8r82\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:54.756460 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.756459 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-trusted-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:54.756578 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.756470 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:54.756578 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.756479 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-service-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:54.756578 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.756488 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-console-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:54.756578 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.756496 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cda45a-cc07-47d1-80a9-371199017ba0-oauth-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:54.756578 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:54.756504 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cda45a-cc07-47d1-80a9-371199017ba0-console-oauth-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:24:55.151828 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:55.151795 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39cda45a-cc07-47d1-80a9-371199017ba0" path="/var/lib/kubelet/pods/39cda45a-cc07-47d1-80a9-371199017ba0/volumes" Jul 08 13:24:57.801604 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:57.801572 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:57.901786 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:57.901751 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:58.003558 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:58.003528 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:24:59.550274 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:24:59.550244 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qkjqn" Jul 08 13:25:05.023453 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:05.023421 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:05.076904 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:05.076881 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:05.679618 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:05.679574 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-697fd7fc84-l76m9" podUID="fb132285-4250-40d6-b1be-545f413ba7fa" containerName="console" containerID="cri-o://5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada" gracePeriod=15 Jul 08 13:25:05.931541 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:05.931489 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:25:06.038204 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038166 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwvhv\" (UniqueName: \"kubernetes.io/projected/fb132285-4250-40d6-b1be-545f413ba7fa-kube-api-access-mwvhv\") pod \"fb132285-4250-40d6-b1be-545f413ba7fa\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038241 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-oauth-serving-cert\") pod \"fb132285-4250-40d6-b1be-545f413ba7fa\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038284 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-console-config\") pod \"fb132285-4250-40d6-b1be-545f413ba7fa\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038366 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-serving-cert\") pod \"fb132285-4250-40d6-b1be-545f413ba7fa\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038405 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-oauth-config\") pod \"fb132285-4250-40d6-b1be-545f413ba7fa\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038455 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-service-ca\") pod \"fb132285-4250-40d6-b1be-545f413ba7fa\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038471 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-trusted-ca-bundle\") pod \"fb132285-4250-40d6-b1be-545f413ba7fa\" (UID: \"fb132285-4250-40d6-b1be-545f413ba7fa\") " Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038609 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-console-config" (OuterVolumeSpecName: "console-config") pod "fb132285-4250-40d6-b1be-545f413ba7fa" (UID: "fb132285-4250-40d6-b1be-545f413ba7fa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:25:06.038653 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038636 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb132285-4250-40d6-b1be-545f413ba7fa" (UID: "fb132285-4250-40d6-b1be-545f413ba7fa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:25:06.039059 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038731 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-oauth-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:25:06.039059 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038748 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-console-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:25:06.039059 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.038822 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb132285-4250-40d6-b1be-545f413ba7fa" (UID: "fb132285-4250-40d6-b1be-545f413ba7fa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:25:06.039059 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.039032 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fb132285-4250-40d6-b1be-545f413ba7fa" (UID: "fb132285-4250-40d6-b1be-545f413ba7fa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:25:06.040677 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.040647 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb132285-4250-40d6-b1be-545f413ba7fa" (UID: "fb132285-4250-40d6-b1be-545f413ba7fa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:25:06.040781 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.040744 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb132285-4250-40d6-b1be-545f413ba7fa" (UID: "fb132285-4250-40d6-b1be-545f413ba7fa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:25:06.040841 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.040826 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb132285-4250-40d6-b1be-545f413ba7fa-kube-api-access-mwvhv" (OuterVolumeSpecName: "kube-api-access-mwvhv") pod "fb132285-4250-40d6-b1be-545f413ba7fa" (UID: "fb132285-4250-40d6-b1be-545f413ba7fa"). InnerVolumeSpecName "kube-api-access-mwvhv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:25:06.139237 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.139209 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-oauth-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:25:06.139237 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.139234 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-service-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:25:06.139237 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.139243 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb132285-4250-40d6-b1be-545f413ba7fa-trusted-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:25:06.139445 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.139252 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwvhv\" (UniqueName: \"kubernetes.io/projected/fb132285-4250-40d6-b1be-545f413ba7fa-kube-api-access-mwvhv\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:25:06.139445 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.139261 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb132285-4250-40d6-b1be-545f413ba7fa-console-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:25:06.719511 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.719459 2560 generic.go:356] "Generic (PLEG): container finished" podID="fb132285-4250-40d6-b1be-545f413ba7fa" containerID="5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada" exitCode=0 Jul 08 13:25:06.719511 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.719507 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697fd7fc84-l76m9" Jul 08 13:25:06.719729 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.719566 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd7fc84-l76m9" event={"ID":"fb132285-4250-40d6-b1be-545f413ba7fa","Type":"ContainerDied","Data":"5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada"} Jul 08 13:25:06.719729 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.719609 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697fd7fc84-l76m9" event={"ID":"fb132285-4250-40d6-b1be-545f413ba7fa","Type":"ContainerDied","Data":"d5be57b76d66243a2fc91c8e48f8951ed52ca73f469f89cc1b551643a504044f"} Jul 08 13:25:06.719729 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.719627 2560 scope.go:117] "RemoveContainer" containerID="5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada" Jul 08 13:25:06.731917 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.731897 2560 scope.go:117] "RemoveContainer" containerID="5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada" Jul 08 13:25:06.732193 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:25:06.732174 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada\": container with ID starting with 5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada not found: ID does not exist" containerID="5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada" Jul 08 13:25:06.732233 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.732201 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada"} err="failed to get container status \"5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada\": rpc error: code = NotFound desc = could not find container \"5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada\": container with ID starting with 5a74e991f06772aed2bb5532fd728a4ce945643f9dd053b1d6d1a6b5416f9ada not found: ID does not exist" Jul 08 13:25:06.743633 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.743608 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697fd7fc84-l76m9"] Jul 08 13:25:06.747071 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:06.747048 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-697fd7fc84-l76m9"] Jul 08 13:25:07.128826 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:07.128786 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb132285-4250-40d6-b1be-545f413ba7fa" path="/var/lib/kubelet/pods/fb132285-4250-40d6-b1be-545f413ba7fa/volumes" Jul 08 13:25:21.907309 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:21.907274 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:25:21.927300 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:21.927277 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:25:22.824219 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:22.824190 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 13:25:31.422312 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:31.422244 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:31.484893 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:31.484842 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:32.472668 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:32.472635 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:43.333803 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:43.333765 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:43.395563 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:43.395528 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:45.565568 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565534 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86748f9699-xhmhx"] Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565826 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="241dd353-adcc-4f84-adc1-4ad357e8cb60" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565836 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="241dd353-adcc-4f84-adc1-4ad357e8cb60" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565853 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39cda45a-cc07-47d1-80a9-371199017ba0" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565859 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39cda45a-cc07-47d1-80a9-371199017ba0" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565873 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb132285-4250-40d6-b1be-545f413ba7fa" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565879 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb132285-4250-40d6-b1be-545f413ba7fa" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565935 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb132285-4250-40d6-b1be-545f413ba7fa" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565946 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="241dd353-adcc-4f84-adc1-4ad357e8cb60" containerName="console" Jul 08 13:25:45.565993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.565952 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="39cda45a-cc07-47d1-80a9-371199017ba0" containerName="console" Jul 08 13:25:45.568036 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.568020 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.581793 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.581760 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86748f9699-xhmhx"] Jul 08 13:25:45.626735 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.626702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:45.659038 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.658998 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-oauth-serving-cert\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.659202 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.659049 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-trusted-ca-bundle\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.659202 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.659075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-serving-cert\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.659202 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.659144 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-console-config\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.659366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.659233 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-service-ca\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.659366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.659279 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9cfq\" (UniqueName: \"kubernetes.io/projected/79c624de-3c49-430c-b796-d56e850c97d6-kube-api-access-w9cfq\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.659366 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.659335 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-oauth-config\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.706492 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.706461 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:45.760448 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.760413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-service-ca\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.760448 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.760455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9cfq\" (UniqueName: \"kubernetes.io/projected/79c624de-3c49-430c-b796-d56e850c97d6-kube-api-access-w9cfq\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.760712 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.760499 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-oauth-config\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.760712 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.760533 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-oauth-serving-cert\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.760712 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.760692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-trusted-ca-bundle\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.760873 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.760748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-serving-cert\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.760873 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.760806 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-console-config\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.761317 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.761294 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-service-ca\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.761446 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.761390 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-oauth-serving-cert\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.761508 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.761481 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-console-config\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.761611 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.761589 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-trusted-ca-bundle\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.763095 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.763078 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-oauth-config\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.763510 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.763493 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-serving-cert\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.768910 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.768880 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9cfq\" (UniqueName: \"kubernetes.io/projected/79c624de-3c49-430c-b796-d56e850c97d6-kube-api-access-w9cfq\") pod \"console-86748f9699-xhmhx\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:45.877032 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:45.876916 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:46.009894 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:46.009860 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86748f9699-xhmhx"] Jul 08 13:25:46.013413 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:25:46.013383 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c624de_3c49_430c_b796_d56e850c97d6.slice/crio-5be999cc7354290d1a52bc5f74d455ba7710cbe694fca00854ac507cec240f0e WatchSource:0}: Error finding container 5be999cc7354290d1a52bc5f74d455ba7710cbe694fca00854ac507cec240f0e: Status 404 returned error can't find the container with id 5be999cc7354290d1a52bc5f74d455ba7710cbe694fca00854ac507cec240f0e Jul 08 13:25:46.839354 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:46.839314 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:46.896884 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:46.896844 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86748f9699-xhmhx" event={"ID":"79c624de-3c49-430c-b796-d56e850c97d6","Type":"ContainerStarted","Data":"9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe"} Jul 08 13:25:46.897084 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:46.896905 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86748f9699-xhmhx" event={"ID":"79c624de-3c49-430c-b796-d56e850c97d6","Type":"ContainerStarted","Data":"5be999cc7354290d1a52bc5f74d455ba7710cbe694fca00854ac507cec240f0e"} Jul 08 13:25:46.913476 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:46.913423 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86748f9699-xhmhx" podStartSLOduration=1.913406004 podStartE2EDuration="1.913406004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:25:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:25:46.91277856 +0000 UTC m=+150.357966232" watchObservedRunningTime="2026-07-08 13:25:46.913406004 +0000 UTC m=+150.358593675" Jul 08 13:25:50.037538 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:50.037498 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:50.138607 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:50.138568 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:50.193828 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:50.193770 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:55.877901 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:55.877860 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:55.878306 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:55.877918 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:55.883451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:55.882947 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:55.936186 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:55.936155 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:25:55.981311 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:55.981274 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c96cc7f4b-pk2mk"] Jul 08 13:25:56.083633 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:56.083598 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:56.136255 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:56.136173 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:25:56.186270 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:25:56.186240 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:06.773562 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:06.773494 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:06.875154 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:06.875122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:21.010088 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.009978 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7c96cc7f4b-pk2mk" podUID="11a4777a-0ee8-4994-9116-bc566e1ea3f9" containerName="console" containerID="cri-o://af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679" gracePeriod=15 Jul 08 13:26:21.248770 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.248748 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:26:21.372332 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372295 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-trusted-ca-bundle\") pod \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " Jul 08 13:26:21.372495 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372345 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-oauth-config\") pod \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " Jul 08 13:26:21.372495 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372369 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-service-ca\") pod \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " Jul 08 13:26:21.372591 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372552 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-config\") pod \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " Jul 08 13:26:21.372642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372590 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-oauth-serving-cert\") pod \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " Jul 08 13:26:21.372642 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372620 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh7h9\" (UniqueName: \"kubernetes.io/projected/11a4777a-0ee8-4994-9116-bc566e1ea3f9-kube-api-access-zh7h9\") pod \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " Jul 08 13:26:21.372741 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372693 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-serving-cert\") pod \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\" (UID: \"11a4777a-0ee8-4994-9116-bc566e1ea3f9\") " Jul 08 13:26:21.372741 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372711 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11a4777a-0ee8-4994-9116-bc566e1ea3f9" (UID: "11a4777a-0ee8-4994-9116-bc566e1ea3f9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:26:21.372898 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372743 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-service-ca" (OuterVolumeSpecName: "service-ca") pod "11a4777a-0ee8-4994-9116-bc566e1ea3f9" (UID: "11a4777a-0ee8-4994-9116-bc566e1ea3f9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:26:21.372981 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372966 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11a4777a-0ee8-4994-9116-bc566e1ea3f9" (UID: "11a4777a-0ee8-4994-9116-bc566e1ea3f9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:26:21.373030 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.372981 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-config" (OuterVolumeSpecName: "console-config") pod "11a4777a-0ee8-4994-9116-bc566e1ea3f9" (UID: "11a4777a-0ee8-4994-9116-bc566e1ea3f9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:26:21.373030 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.373001 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-trusted-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:26:21.373030 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.373018 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-service-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:26:21.374687 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.374660 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11a4777a-0ee8-4994-9116-bc566e1ea3f9" (UID: "11a4777a-0ee8-4994-9116-bc566e1ea3f9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:26:21.374794 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.374715 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a4777a-0ee8-4994-9116-bc566e1ea3f9-kube-api-access-zh7h9" (OuterVolumeSpecName: "kube-api-access-zh7h9") pod "11a4777a-0ee8-4994-9116-bc566e1ea3f9" (UID: "11a4777a-0ee8-4994-9116-bc566e1ea3f9"). InnerVolumeSpecName "kube-api-access-zh7h9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:26:21.374836 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.374788 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11a4777a-0ee8-4994-9116-bc566e1ea3f9" (UID: "11a4777a-0ee8-4994-9116-bc566e1ea3f9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:26:21.473406 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.473371 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-oauth-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:26:21.473406 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.473400 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:26:21.473583 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.473414 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11a4777a-0ee8-4994-9116-bc566e1ea3f9-oauth-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:26:21.473583 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.473426 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh7h9\" (UniqueName: \"kubernetes.io/projected/11a4777a-0ee8-4994-9116-bc566e1ea3f9-kube-api-access-zh7h9\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:26:21.473583 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.473438 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11a4777a-0ee8-4994-9116-bc566e1ea3f9-console-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:26:21.612814 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:21.612777 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:22.036540 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.036500 2560 generic.go:356] "Generic (PLEG): container finished" podID="11a4777a-0ee8-4994-9116-bc566e1ea3f9" containerID="af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679" exitCode=0 Jul 08 13:26:22.036979 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.036554 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c96cc7f4b-pk2mk" Jul 08 13:26:22.036979 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.036586 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c96cc7f4b-pk2mk" event={"ID":"11a4777a-0ee8-4994-9116-bc566e1ea3f9","Type":"ContainerDied","Data":"af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679"} Jul 08 13:26:22.036979 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.036640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c96cc7f4b-pk2mk" event={"ID":"11a4777a-0ee8-4994-9116-bc566e1ea3f9","Type":"ContainerDied","Data":"9945584b2fcf78f34ae8d37d41405461043ad5508b49211efcd3d3bfe4d7bb71"} Jul 08 13:26:22.036979 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.036665 2560 scope.go:117] "RemoveContainer" containerID="af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679" Jul 08 13:26:22.045491 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.045477 2560 scope.go:117] "RemoveContainer" containerID="af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679" Jul 08 13:26:22.045762 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:26:22.045743 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679\": container with ID starting with af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679 not found: ID does not exist" containerID="af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679" Jul 08 13:26:22.045811 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.045770 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679"} err="failed to get container status \"af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679\": rpc error: code = NotFound desc = could not find container \"af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679\": container with ID starting with af027900544aa77e654e1518b2a2722899447262fd4084d5eb5796c7fd82a679 not found: ID does not exist" Jul 08 13:26:22.056209 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.056182 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c96cc7f4b-pk2mk"] Jul 08 13:26:22.058558 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:22.058537 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c96cc7f4b-pk2mk"] Jul 08 13:26:23.129257 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:23.129219 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11a4777a-0ee8-4994-9116-bc566e1ea3f9" path="/var/lib/kubelet/pods/11a4777a-0ee8-4994-9116-bc566e1ea3f9/volumes" Jul 08 13:26:33.374533 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:33.374494 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:34.198786 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:34.198743 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:36.984769 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:36.984732 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:37.084953 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:37.084909 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:38.151848 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:38.151816 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:38.263572 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:38.263531 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:38.935154 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:38.935117 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:39.031342 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:39.031292 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:41.103701 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:41.103658 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:41.298575 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:41.298536 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:43.819125 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:43.819087 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:43.917448 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:43.917414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:49.309092 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:49.309054 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:49.784786 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:49.784704 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:49.885218 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:49.885187 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:52.011164 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.011131 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h"] Jul 08 13:26:52.011554 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.011430 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11a4777a-0ee8-4994-9116-bc566e1ea3f9" containerName="console" Jul 08 13:26:52.011554 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.011440 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="11a4777a-0ee8-4994-9116-bc566e1ea3f9" containerName="console" Jul 08 13:26:52.011554 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.011486 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="11a4777a-0ee8-4994-9116-bc566e1ea3f9" containerName="console" Jul 08 13:26:52.013376 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.013358 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.015755 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.015736 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-h9ddc\"" Jul 08 13:26:52.015870 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.015762 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 13:26:52.016582 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.016566 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 13:26:52.027060 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.027040 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h"] Jul 08 13:26:52.119883 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.119851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nx5s\" (UniqueName: \"kubernetes.io/projected/4643b9d4-ed8f-4536-b559-d28070095272-kube-api-access-6nx5s\") pod \"openshift-lws-operator-fd8ccff4c-72d8h\" (UID: \"4643b9d4-ed8f-4536-b559-d28070095272\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.120063 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.119890 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4643b9d4-ed8f-4536-b559-d28070095272-tmp\") pod \"openshift-lws-operator-fd8ccff4c-72d8h\" (UID: \"4643b9d4-ed8f-4536-b559-d28070095272\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.220775 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.220730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nx5s\" (UniqueName: \"kubernetes.io/projected/4643b9d4-ed8f-4536-b559-d28070095272-kube-api-access-6nx5s\") pod \"openshift-lws-operator-fd8ccff4c-72d8h\" (UID: \"4643b9d4-ed8f-4536-b559-d28070095272\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.220775 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.220790 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4643b9d4-ed8f-4536-b559-d28070095272-tmp\") pod \"openshift-lws-operator-fd8ccff4c-72d8h\" (UID: \"4643b9d4-ed8f-4536-b559-d28070095272\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.221228 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.221208 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4643b9d4-ed8f-4536-b559-d28070095272-tmp\") pod \"openshift-lws-operator-fd8ccff4c-72d8h\" (UID: \"4643b9d4-ed8f-4536-b559-d28070095272\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.228766 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.228738 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nx5s\" (UniqueName: \"kubernetes.io/projected/4643b9d4-ed8f-4536-b559-d28070095272-kube-api-access-6nx5s\") pod \"openshift-lws-operator-fd8ccff4c-72d8h\" (UID: \"4643b9d4-ed8f-4536-b559-d28070095272\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.330821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.330792 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" Jul 08 13:26:52.456716 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.456692 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h"] Jul 08 13:26:52.459172 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:26:52.459143 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4643b9d4_ed8f_4536_b559_d28070095272.slice/crio-e969d4c0bd86c9f256ce2adf8aaa17911d701b5317d1aeb711b4b519fdb7c542 WatchSource:0}: Error finding container e969d4c0bd86c9f256ce2adf8aaa17911d701b5317d1aeb711b4b519fdb7c542: Status 404 returned error can't find the container with id e969d4c0bd86c9f256ce2adf8aaa17911d701b5317d1aeb711b4b519fdb7c542 Jul 08 13:26:52.994332 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:52.994299 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:53.151671 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:53.151624 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" event={"ID":"4643b9d4-ed8f-4536-b559-d28070095272","Type":"ContainerStarted","Data":"e969d4c0bd86c9f256ce2adf8aaa17911d701b5317d1aeb711b4b519fdb7c542"} Jul 08 13:26:55.160768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:55.160725 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" event={"ID":"4643b9d4-ed8f-4536-b559-d28070095272","Type":"ContainerStarted","Data":"091e49aadd267a23d3ef9be109afae10ac4a9ee0aa993f95067bbe0f04cbbc21"} Jul 08 13:26:55.178332 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:55.178278 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72d8h" podStartSLOduration=1.837141188 podStartE2EDuration="4.178259511s" totalImagesPullingTime="2.341118323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:26:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:26:55.17734847 +0000 UTC m=+218.622536142" watchObservedRunningTime="2026-07-08 13:26:55.178259511 +0000 UTC m=+218.623447183" Jul 08 13:26:55.842388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:55.842354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:55.946990 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:55.946950 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:26:57.978142 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:26:57.978106 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:07.455291 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:07.455236 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:07.508520 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:07.508491 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:12.312547 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.312519 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:12.354303 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.354272 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24"] Jul 08 13:27:12.357604 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.357586 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.361340 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.361323 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lk7ng\"" Jul 08 13:27:12.362295 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.362269 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 13:27:12.362392 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.362366 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 13:27:12.362461 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.362412 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 13:27:12.395852 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.395830 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24"] Jul 08 13:27:12.434769 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.434734 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:12.501884 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.501858 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8bba91-ff90-4e50-b781-07c43c7ae02e-cert\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.502005 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.501893 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtrrj\" (UniqueName: \"kubernetes.io/projected/bd8bba91-ff90-4e50-b781-07c43c7ae02e-kube-api-access-dtrrj\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.502005 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.501949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bd8bba91-ff90-4e50-b781-07c43c7ae02e-manager-config\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.502005 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.501981 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd8bba91-ff90-4e50-b781-07c43c7ae02e-metrics-cert\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.602962 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.602884 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bd8bba91-ff90-4e50-b781-07c43c7ae02e-manager-config\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.602962 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.602943 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd8bba91-ff90-4e50-b781-07c43c7ae02e-metrics-cert\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.603094 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.603002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8bba91-ff90-4e50-b781-07c43c7ae02e-cert\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.603094 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.603037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtrrj\" (UniqueName: \"kubernetes.io/projected/bd8bba91-ff90-4e50-b781-07c43c7ae02e-kube-api-access-dtrrj\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.603590 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.603572 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/bd8bba91-ff90-4e50-b781-07c43c7ae02e-manager-config\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.605553 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.605536 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/bd8bba91-ff90-4e50-b781-07c43c7ae02e-metrics-cert\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.605692 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.605674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd8bba91-ff90-4e50-b781-07c43c7ae02e-cert\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.614201 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.614182 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtrrj\" (UniqueName: \"kubernetes.io/projected/bd8bba91-ff90-4e50-b781-07c43c7ae02e-kube-api-access-dtrrj\") pod \"lws-controller-manager-7fcdcfc8c4-t8g24\" (UID: \"bd8bba91-ff90-4e50-b781-07c43c7ae02e\") " pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.666587 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.666566 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:12.803467 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:12.803440 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24"] Jul 08 13:27:12.805373 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:27:12.805345 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd8bba91_ff90_4e50_b781_07c43c7ae02e.slice/crio-5c872b1bd9e48264c5354639e2fc25cbefe1b52abb2b1600c2da62ae617cee5b WatchSource:0}: Error finding container 5c872b1bd9e48264c5354639e2fc25cbefe1b52abb2b1600c2da62ae617cee5b: Status 404 returned error can't find the container with id 5c872b1bd9e48264c5354639e2fc25cbefe1b52abb2b1600c2da62ae617cee5b Jul 08 13:27:13.144983 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:13.144948 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:13.232781 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:13.232749 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" event={"ID":"bd8bba91-ff90-4e50-b781-07c43c7ae02e","Type":"ContainerStarted","Data":"5c872b1bd9e48264c5354639e2fc25cbefe1b52abb2b1600c2da62ae617cee5b"} Jul 08 13:27:13.237302 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:13.237284 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:13.740908 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:13.740874 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:13.989986 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:13.989896 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:14.096368 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.096336 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:14.148171 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.148138 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj"] Jul 08 13:27:14.151894 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.151870 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.156428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.156014 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hjkcl\"" Jul 08 13:27:14.156428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.156032 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 13:27:14.156428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.156070 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 13:27:14.156428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.156106 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 13:27:14.156428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.156014 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 13:27:14.202319 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.202256 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj"] Jul 08 13:27:14.318466 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.318434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2769322-e620-458e-804e-0407f9efea88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.318646 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.318477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2769322-e620-458e-804e-0407f9efea88-webhook-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.318646 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.318559 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2769322-e620-458e-804e-0407f9efea88-manifests\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.318646 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.318627 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2769322-e620-458e-804e-0407f9efea88-tmp\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.318818 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.318680 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn4vq\" (UniqueName: \"kubernetes.io/projected/f2769322-e620-458e-804e-0407f9efea88-kube-api-access-pn4vq\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.325403 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.325370 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:14.420102 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.420035 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2769322-e620-458e-804e-0407f9efea88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.420102 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.420072 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2769322-e620-458e-804e-0407f9efea88-webhook-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.420102 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.420099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2769322-e620-458e-804e-0407f9efea88-manifests\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.420308 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.420216 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2769322-e620-458e-804e-0407f9efea88-tmp\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.420308 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.420266 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn4vq\" (UniqueName: \"kubernetes.io/projected/f2769322-e620-458e-804e-0407f9efea88-kube-api-access-pn4vq\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.420424 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.420376 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2769322-e620-458e-804e-0407f9efea88-manifests\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.422587 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.422556 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2769322-e620-458e-804e-0407f9efea88-tmp\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.422587 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.422577 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2769322-e620-458e-804e-0407f9efea88-webhook-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.422746 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.422631 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2769322-e620-458e-804e-0407f9efea88-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.425845 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.425806 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:14.429075 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.429051 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn4vq\" (UniqueName: \"kubernetes.io/projected/f2769322-e620-458e-804e-0407f9efea88-kube-api-access-pn4vq\") pod \"opendatahub-operator-controller-manager-774b77d499-6thdj\" (UID: \"f2769322-e620-458e-804e-0407f9efea88\") " pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.466268 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.466249 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:14.527437 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.527087 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:14.642221 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.642187 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj"] Jul 08 13:27:14.647031 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.647002 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:14.647213 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:27:14.647187 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2769322_e620_458e_804e_0407f9efea88.slice/crio-c568cfc907ad411337b43f1fdcdff70689bd32005444fbb118760aedb09c63ca WatchSource:0}: Error finding container c568cfc907ad411337b43f1fdcdff70689bd32005444fbb118760aedb09c63ca: Status 404 returned error can't find the container with id c568cfc907ad411337b43f1fdcdff70689bd32005444fbb118760aedb09c63ca Jul 08 13:27:14.746980 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.746876 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:14.852445 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:14.852403 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:15.244720 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:15.244687 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" event={"ID":"f2769322-e620-458e-804e-0407f9efea88","Type":"ContainerStarted","Data":"c568cfc907ad411337b43f1fdcdff70689bd32005444fbb118760aedb09c63ca"} Jul 08 13:27:15.246679 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:15.246648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" event={"ID":"bd8bba91-ff90-4e50-b781-07c43c7ae02e","Type":"ContainerStarted","Data":"cdb670f7ce7405d3c02bd12b31be55195ad92ebdaff65e2510fe522ec995d113"} Jul 08 13:27:15.246862 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:15.246843 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:15.265640 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:15.265591 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" podStartSLOduration=1.633110393 podStartE2EDuration="3.265577713s" totalImagesPullingTime="1.63246732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:15.263844427 +0000 UTC m=+238.709032098" watchObservedRunningTime="2026-07-08 13:27:15.265577713 +0000 UTC m=+238.710765382" Jul 08 13:27:18.268869 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:18.268783 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" event={"ID":"f2769322-e620-458e-804e-0407f9efea88","Type":"ContainerStarted","Data":"6b2a71b658f9981d9fd5bcacfe48f1254f1cc11e8ab4ab223fced13d821a6e44"} Jul 08 13:27:19.274656 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:19.274617 2560 generic.go:356] "Generic (PLEG): container finished" podID="f2769322-e620-458e-804e-0407f9efea88" containerID="6b2a71b658f9981d9fd5bcacfe48f1254f1cc11e8ab4ab223fced13d821a6e44" exitCode=0 Jul 08 13:27:19.275141 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:19.274691 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" event={"ID":"f2769322-e620-458e-804e-0407f9efea88","Type":"ContainerDied","Data":"6b2a71b658f9981d9fd5bcacfe48f1254f1cc11e8ab4ab223fced13d821a6e44"} Jul 08 13:27:20.282087 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:20.282044 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" event={"ID":"f2769322-e620-458e-804e-0407f9efea88","Type":"ContainerStarted","Data":"3dff130586c0c9c0a43bf9611c8bb44f157d9a8c716b1e4fec2b86eb5d283124"} Jul 08 13:27:20.282487 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:20.282192 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:20.306126 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:20.306066 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" podStartSLOduration=1.65543788 podStartE2EDuration="6.306047394s" totalImagesPullingTime="3.611690825s" totalInitContainerRuntime="1.038918689s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:20.304236995 +0000 UTC m=+243.749424666" watchObservedRunningTime="2026-07-08 13:27:20.306047394 +0000 UTC m=+243.751235063" Jul 08 13:27:26.256754 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:26.256726 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7fcdcfc8c4-t8g24" Jul 08 13:27:31.288597 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:31.288569 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-774b77d499-6thdj" Jul 08 13:27:33.730914 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:33.730881 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:33.839726 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:33.839688 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:33.959244 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:33.959178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:34.006819 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.006738 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-746f76655b-glgm9"] Jul 08 13:27:34.014972 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.014945 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.019616 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.019594 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 13:27:34.019873 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.019854 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rlwtc\"" Jul 08 13:27:34.020001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.019907 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 13:27:34.020001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.019949 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 13:27:34.020001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.019910 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 13:27:34.024241 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.024216 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-746f76655b-glgm9"] Jul 08 13:27:34.083971 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.083918 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e3da55b-5ba1-4d72-8372-388176e40522-tmp\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.084115 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.084003 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfvc8\" (UniqueName: \"kubernetes.io/projected/1e3da55b-5ba1-4d72-8372-388176e40522-kube-api-access-kfvc8\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.084115 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.084032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e3da55b-5ba1-4d72-8372-388176e40522-tls-certs\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.084466 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.084440 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:34.185198 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.185166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e3da55b-5ba1-4d72-8372-388176e40522-tmp\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.185348 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.185213 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfvc8\" (UniqueName: \"kubernetes.io/projected/1e3da55b-5ba1-4d72-8372-388176e40522-kube-api-access-kfvc8\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.185348 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.185231 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e3da55b-5ba1-4d72-8372-388176e40522-tls-certs\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.187477 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.187452 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e3da55b-5ba1-4d72-8372-388176e40522-tmp\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.187782 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.187761 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e3da55b-5ba1-4d72-8372-388176e40522-tls-certs\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.200690 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.200667 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfvc8\" (UniqueName: \"kubernetes.io/projected/1e3da55b-5ba1-4d72-8372-388176e40522-kube-api-access-kfvc8\") pod \"kube-auth-proxy-746f76655b-glgm9\" (UID: \"1e3da55b-5ba1-4d72-8372-388176e40522\") " pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.328446 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.328416 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" Jul 08 13:27:34.420387 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.420344 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:34.468813 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.468788 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-746f76655b-glgm9"] Jul 08 13:27:34.470559 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:27:34.470533 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3da55b_5ba1_4d72_8372_388176e40522.slice/crio-d3c4a821b2a2fb2af05a66240a0f5b311059abdb631fb71ca62e643679374860 WatchSource:0}: Error finding container d3c4a821b2a2fb2af05a66240a0f5b311059abdb631fb71ca62e643679374860: Status 404 returned error can't find the container with id d3c4a821b2a2fb2af05a66240a0f5b311059abdb631fb71ca62e643679374860 Jul 08 13:27:34.524471 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.524444 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:34.619802 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:34.619715 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:35.149615 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:35.149577 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:35.351893 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:35.351844 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" event={"ID":"1e3da55b-5ba1-4d72-8372-388176e40522","Type":"ContainerStarted","Data":"d3c4a821b2a2fb2af05a66240a0f5b311059abdb631fb71ca62e643679374860"} Jul 08 13:27:35.365633 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:35.365599 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:35.458109 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:35.457988 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:35.751608 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:35.751286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:36.371651 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:36.371620 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:38.367186 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:38.367151 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" event={"ID":"1e3da55b-5ba1-4d72-8372-388176e40522","Type":"ContainerStarted","Data":"1fe6d559d865d062599db68c4f90e64540f286786488cfeda056fdf5deda9d4a"} Jul 08 13:27:38.386286 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:38.386213 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-746f76655b-glgm9" podStartSLOduration=2.098845753 podStartE2EDuration="5.386194891s" totalImagesPullingTime="3.287349138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:38.384903766 +0000 UTC m=+261.830091438" watchObservedRunningTime="2026-07-08 13:27:38.386194891 +0000 UTC m=+261.831382611" Jul 08 13:27:41.377450 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:41.377416 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:41.475824 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:41.475783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:42.142767 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:42.142733 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:42.251487 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:42.251452 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:42.483475 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:42.483388 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:44.811124 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.811091 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:44.868862 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.868825 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg"] Jul 08 13:27:44.874286 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.874265 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:44.876586 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.876565 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 08 13:27:44.876681 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.876587 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qgmhd\"" Jul 08 13:27:44.882538 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.882516 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg"] Jul 08 13:27:44.985354 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.985326 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldk6l\" (UniqueName: \"kubernetes.io/projected/028c363b-2646-4686-b821-4847944bf71d-kube-api-access-ldk6l\") pod \"llmisvc-controller-manager-777c6bd7d5-w8rsg\" (UID: \"028c363b-2646-4686-b821-4847944bf71d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:44.985505 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:44.985361 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/028c363b-2646-4686-b821-4847944bf71d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w8rsg\" (UID: \"028c363b-2646-4686-b821-4847944bf71d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:45.086623 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:45.086546 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/028c363b-2646-4686-b821-4847944bf71d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w8rsg\" (UID: \"028c363b-2646-4686-b821-4847944bf71d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:45.086802 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:45.086629 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldk6l\" (UniqueName: \"kubernetes.io/projected/028c363b-2646-4686-b821-4847944bf71d-kube-api-access-ldk6l\") pod \"llmisvc-controller-manager-777c6bd7d5-w8rsg\" (UID: \"028c363b-2646-4686-b821-4847944bf71d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:45.086802 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:27:45.086732 2560 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 08 13:27:45.087016 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:27:45.086823 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/028c363b-2646-4686-b821-4847944bf71d-cert podName:028c363b-2646-4686-b821-4847944bf71d nodeName:}" failed. No retries permitted until 2026-07-08 13:27:45.586800027 +0000 UTC m=+269.031987689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/028c363b-2646-4686-b821-4847944bf71d-cert") pod "llmisvc-controller-manager-777c6bd7d5-w8rsg" (UID: "028c363b-2646-4686-b821-4847944bf71d") : secret "llmisvc-webhook-server-cert" not found Jul 08 13:27:45.100344 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:45.100319 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldk6l\" (UniqueName: \"kubernetes.io/projected/028c363b-2646-4686-b821-4847944bf71d-kube-api-access-ldk6l\") pod \"llmisvc-controller-manager-777c6bd7d5-w8rsg\" (UID: \"028c363b-2646-4686-b821-4847944bf71d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:45.591708 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:45.591676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/028c363b-2646-4686-b821-4847944bf71d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w8rsg\" (UID: \"028c363b-2646-4686-b821-4847944bf71d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:45.594138 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:45.594110 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/028c363b-2646-4686-b821-4847944bf71d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w8rsg\" (UID: \"028c363b-2646-4686-b821-4847944bf71d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:45.786238 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:45.786195 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:45.913401 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:45.913372 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg"] Jul 08 13:27:45.916212 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:27:45.916186 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod028c363b_2646_4686_b821_4847944bf71d.slice/crio-fe0aac9112731d4a12b0bccc3a131859c5aa2c72a0d7b36004d4de08d9359e00 WatchSource:0}: Error finding container fe0aac9112731d4a12b0bccc3a131859c5aa2c72a0d7b36004d4de08d9359e00: Status 404 returned error can't find the container with id fe0aac9112731d4a12b0bccc3a131859c5aa2c72a0d7b36004d4de08d9359e00 Jul 08 13:27:46.108661 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:46.108626 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:46.205325 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:46.205241 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:46.305260 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:46.305231 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:27:46.403680 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:46.403648 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" event={"ID":"028c363b-2646-4686-b821-4847944bf71d","Type":"ContainerStarted","Data":"fe0aac9112731d4a12b0bccc3a131859c5aa2c72a0d7b36004d4de08d9359e00"} Jul 08 13:27:49.419632 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:49.419539 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" event={"ID":"028c363b-2646-4686-b821-4847944bf71d","Type":"ContainerStarted","Data":"fd2f0136ec026b73c29a068c5a414480b850c982c21dfed065fc96df45b3dc22"} Jul 08 13:27:49.420030 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:49.419664 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:27:49.441558 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:27:49.441494 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" podStartSLOduration=2.204550515 podStartE2EDuration="5.44147935s" totalImagesPullingTime="3.236928835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:27:49.439048899 +0000 UTC m=+272.884236570" watchObservedRunningTime="2026-07-08 13:27:49.44147935 +0000 UTC m=+272.886667020" Jul 08 13:28:16.934758 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:16.934729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:28:16.935351 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:16.935226 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:28:16.942354 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:16.942336 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 13:28:20.426253 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:20.426222 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w8rsg" Jul 08 13:28:35.221032 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:35.220991 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:28:35.323747 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:35.323711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:28:35.512394 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:35.512309 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:28:36.144099 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:36.144064 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:28:36.343754 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:28:36.343723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:03.871737 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:03.871702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:04.467380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:04.467340 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:08.872051 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:08.871970 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:11.130402 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:11.130371 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:13.780594 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:13.780546 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:14.088052 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:14.088016 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:16.469228 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:16.469193 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:17.126325 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:17.126265 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:19.550074 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:19.550042 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:20.281218 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:20.281171 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:20.382443 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:20.382405 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:20.782353 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:20.782315 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:20.879480 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:20.879446 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:22.922513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.922478 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg"] Jul 08 13:29:22.925760 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.925740 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:22.928552 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.928529 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 13:29:22.928706 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.928660 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4zqwg\"" Jul 08 13:29:22.929790 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.929770 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 13:29:22.929897 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.929793 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 13:29:22.929897 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.929769 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 13:29:22.938093 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.938069 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg"] Jul 08 13:29:22.940049 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:22.940025 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:23.018194 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.018162 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a35435b-824e-4f94-b8df-5f45a44d951f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.018194 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.018192 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nzj9\" (UniqueName: \"kubernetes.io/projected/3a35435b-824e-4f94-b8df-5f45a44d951f-kube-api-access-9nzj9\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.018387 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.018220 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3a35435b-824e-4f94-b8df-5f45a44d951f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.034316 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.034286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:23.118839 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.118809 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a35435b-824e-4f94-b8df-5f45a44d951f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.118839 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.118842 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nzj9\" (UniqueName: \"kubernetes.io/projected/3a35435b-824e-4f94-b8df-5f45a44d951f-kube-api-access-9nzj9\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.119075 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.118888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3a35435b-824e-4f94-b8df-5f45a44d951f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.119075 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:29:23.118991 2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 08 13:29:23.119075 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:29:23.119072 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a35435b-824e-4f94-b8df-5f45a44d951f-plugin-serving-cert podName:3a35435b-824e-4f94-b8df-5f45a44d951f nodeName:}" failed. No retries permitted until 2026-07-08 13:29:23.619052516 +0000 UTC m=+367.064240188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/3a35435b-824e-4f94-b8df-5f45a44d951f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-mmvrg" (UID: "3a35435b-824e-4f94-b8df-5f45a44d951f") : secret "plugin-serving-cert" not found Jul 08 13:29:23.119609 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.119591 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3a35435b-824e-4f94-b8df-5f45a44d951f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.130266 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.130243 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nzj9\" (UniqueName: \"kubernetes.io/projected/3a35435b-824e-4f94-b8df-5f45a44d951f-kube-api-access-9nzj9\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.303794 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.303754 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:23.623195 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.623107 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a35435b-824e-4f94-b8df-5f45a44d951f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.625596 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.625568 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a35435b-824e-4f94-b8df-5f45a44d951f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-mmvrg\" (UID: \"3a35435b-824e-4f94-b8df-5f45a44d951f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.834580 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.834551 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" Jul 08 13:29:23.958291 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.958269 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg"] Jul 08 13:29:23.960327 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:29:23.960303 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a35435b_824e_4f94_b8df_5f45a44d951f.slice/crio-296b0bdc7b9bf5465c56913725eeb727fe24236830972c25fd374e648d265f3f WatchSource:0}: Error finding container 296b0bdc7b9bf5465c56913725eeb727fe24236830972c25fd374e648d265f3f: Status 404 returned error can't find the container with id 296b0bdc7b9bf5465c56913725eeb727fe24236830972c25fd374e648d265f3f Jul 08 13:29:23.961798 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:23.961777 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 13:29:24.658779 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:24.658747 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:24.750615 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:24.750578 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:24.831331 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:24.831289 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" event={"ID":"3a35435b-824e-4f94-b8df-5f45a44d951f","Type":"ContainerStarted","Data":"296b0bdc7b9bf5465c56913725eeb727fe24236830972c25fd374e648d265f3f"} Jul 08 13:29:24.850458 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:24.850418 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:24.951032 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:24.950944 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:31.598619 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:31.598580 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:31.722675 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:31.722637 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:33.720381 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.720345 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9"] Jul 08 13:29:33.729811 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.729784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:33.732749 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.732723 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h4s79\"" Jul 08 13:29:33.750340 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.750314 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9"] Jul 08 13:29:33.826810 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.826774 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8nvq\" (UniqueName: \"kubernetes.io/projected/6a46f135-c9aa-4c6d-ae96-009046e88777-kube-api-access-n8nvq\") pod \"kuadrant-operator-controller-manager-86878c769f-k6kj9\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:33.827010 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.826839 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a46f135-c9aa-4c6d-ae96-009046e88777-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k6kj9\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:33.928268 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.928224 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8nvq\" (UniqueName: \"kubernetes.io/projected/6a46f135-c9aa-4c6d-ae96-009046e88777-kube-api-access-n8nvq\") pod \"kuadrant-operator-controller-manager-86878c769f-k6kj9\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:33.928455 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.928293 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a46f135-c9aa-4c6d-ae96-009046e88777-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k6kj9\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:33.928738 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.928700 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a46f135-c9aa-4c6d-ae96-009046e88777-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k6kj9\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:33.937501 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:33.937480 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8nvq\" (UniqueName: \"kubernetes.io/projected/6a46f135-c9aa-4c6d-ae96-009046e88777-kube-api-access-n8nvq\") pod \"kuadrant-operator-controller-manager-86878c769f-k6kj9\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:34.042577 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:34.042500 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:35.510597 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:35.510524 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:35.608552 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:35.608495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:39.307830 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:39.307790 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:39.367138 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:39.367084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:40.206908 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:40.206876 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:40.317062 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:40.317030 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:44.084206 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:44.084168 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:44.186220 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:44.186164 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:47.216610 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:47.216584 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9"] Jul 08 13:29:47.218616 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:29:47.218565 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a46f135_c9aa_4c6d_ae96_009046e88777.slice/crio-8f631e90c0631a1a0ff97a2fda3ae3ca23f6711fc7b0883d71fcce51e314b076 WatchSource:0}: Error finding container 8f631e90c0631a1a0ff97a2fda3ae3ca23f6711fc7b0883d71fcce51e314b076: Status 404 returned error can't find the container with id 8f631e90c0631a1a0ff97a2fda3ae3ca23f6711fc7b0883d71fcce51e314b076 Jul 08 13:29:47.950384 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:47.950341 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" event={"ID":"6a46f135-c9aa-4c6d-ae96-009046e88777","Type":"ContainerStarted","Data":"8f631e90c0631a1a0ff97a2fda3ae3ca23f6711fc7b0883d71fcce51e314b076"} Jul 08 13:29:47.951739 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:47.951713 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" event={"ID":"3a35435b-824e-4f94-b8df-5f45a44d951f","Type":"ContainerStarted","Data":"a7f13bf5f1fc2365654f0899790dfa1a91cf6476cead8a06e4e0158164d35644"} Jul 08 13:29:47.966801 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:47.966750 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-mmvrg" podStartSLOduration=2.423942788 podStartE2EDuration="25.966732525s" totalImagesPullingTime="23.542789737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:29:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:29:47.965963759 +0000 UTC m=+391.411151429" watchObservedRunningTime="2026-07-08 13:29:47.966732525 +0000 UTC m=+391.411920197" Jul 08 13:29:50.507814 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.507782 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c97db7bdf-cthh6"] Jul 08 13:29:50.525520 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.525485 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c97db7bdf-cthh6"] Jul 08 13:29:50.525689 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.525640 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.573751 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.573702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:50.593842 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.593804 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-trusted-ca-bundle\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.593842 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.593847 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7cqb\" (UniqueName: \"kubernetes.io/projected/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-kube-api-access-x7cqb\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.594108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.593879 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-oauth-serving-cert\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.594108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.594000 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-service-ca\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.594108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.594036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-config\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.594108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.594061 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-serving-cert\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.594302 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.594161 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-oauth-config\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.637425 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.637359 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:50.695544 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.695507 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-serving-cert\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.695733 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.695606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-oauth-config\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.695733 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.695664 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-trusted-ca-bundle\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.695733 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.695684 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7cqb\" (UniqueName: \"kubernetes.io/projected/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-kube-api-access-x7cqb\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.695733 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.695702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-oauth-serving-cert\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.695733 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.695733 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-service-ca\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.696052 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.695759 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-config\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.696593 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.696532 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-config\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.696908 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.696786 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-service-ca\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.696908 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.696876 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-oauth-serving-cert\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.697096 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.697069 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-trusted-ca-bundle\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.698661 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.698639 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-serving-cert\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.698834 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.698813 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-console-oauth-config\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.707219 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.707145 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7cqb\" (UniqueName: \"kubernetes.io/projected/d2df0b45-4886-4a96-8bea-6216c3a4c3fa-kube-api-access-x7cqb\") pod \"console-7c97db7bdf-cthh6\" (UID: \"d2df0b45-4886-4a96-8bea-6216c3a4c3fa\") " pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.838463 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.838420 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:29:50.997008 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:50.996830 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c97db7bdf-cthh6"] Jul 08 13:29:51.000409 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:29:51.000379 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2df0b45_4886_4a96_8bea_6216c3a4c3fa.slice/crio-6592078df822a78a1e624d7072218ae029e611ab96611b498a2c6b289aa6fdc3 WatchSource:0}: Error finding container 6592078df822a78a1e624d7072218ae029e611ab96611b498a2c6b289aa6fdc3: Status 404 returned error can't find the container with id 6592078df822a78a1e624d7072218ae029e611ab96611b498a2c6b289aa6fdc3 Jul 08 13:29:51.973551 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:51.973507 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c97db7bdf-cthh6" event={"ID":"d2df0b45-4886-4a96-8bea-6216c3a4c3fa","Type":"ContainerStarted","Data":"7949c1605a6b6b25212e2da4586b7d80bf097efef09affe895b84828c35f238c"} Jul 08 13:29:51.974033 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:51.973571 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c97db7bdf-cthh6" event={"ID":"d2df0b45-4886-4a96-8bea-6216c3a4c3fa","Type":"ContainerStarted","Data":"6592078df822a78a1e624d7072218ae029e611ab96611b498a2c6b289aa6fdc3"} Jul 08 13:29:51.992628 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:51.992573 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c97db7bdf-cthh6" podStartSLOduration=1.99255681 podStartE2EDuration="1.99255681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:29:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:29:51.990502846 +0000 UTC m=+395.435690516" watchObservedRunningTime="2026-07-08 13:29:51.99255681 +0000 UTC m=+395.437744479" Jul 08 13:29:52.286032 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:52.285951 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:52.386478 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:52.386439 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:54.991681 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:54.991582 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" event={"ID":"6a46f135-c9aa-4c6d-ae96-009046e88777","Type":"ContainerStarted","Data":"90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948"} Jul 08 13:29:54.992091 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:54.991726 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:29:55.011340 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:55.011278 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" podStartSLOduration=14.532278561 podStartE2EDuration="22.011256524s" totalImagesPullingTime="7.478977963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:29:55.009766699 +0000 UTC m=+398.454954369" watchObservedRunningTime="2026-07-08 13:29:55.011256524 +0000 UTC m=+398.456444196" Jul 08 13:29:56.535122 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:56.535085 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:56.635984 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:56.635946 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:59.058617 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:59.058581 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:29:59.159118 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:29:59.159082 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:00.487918 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:00.487881 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:00.542677 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:00.542640 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:00.838774 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:00.838732 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:30:00.839017 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:00.838801 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:30:00.843952 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:00.843899 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:30:00.893944 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:00.893888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:00.949279 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:00.949239 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:01.022679 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:01.022648 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c97db7bdf-cthh6" Jul 08 13:30:01.073759 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:01.073724 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86748f9699-xhmhx"] Jul 08 13:30:01.256253 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:01.256161 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:01.375541 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:01.375495 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:04.489825 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:04.489791 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:04.557865 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:04.557824 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:05.998422 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:05.998394 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:30:10.419325 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:10.419286 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:10.477385 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:10.477349 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:15.139396 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:15.139364 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:15.239743 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:15.239702 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:22.458277 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:22.458233 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:22.801686 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:22.801656 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:26.095706 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.095660 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86748f9699-xhmhx" podUID="79c624de-3c49-430c-b796-d56e850c97d6" containerName="console" containerID="cri-o://9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe" gracePeriod=15 Jul 08 13:30:26.356549 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.356485 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:30:26.425195 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425161 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-oauth-config\") pod \"79c624de-3c49-430c-b796-d56e850c97d6\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " Jul 08 13:30:26.425374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425214 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-serving-cert\") pod \"79c624de-3c49-430c-b796-d56e850c97d6\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " Jul 08 13:30:26.425374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425231 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-console-config\") pod \"79c624de-3c49-430c-b796-d56e850c97d6\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " Jul 08 13:30:26.425374 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425252 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-trusted-ca-bundle\") pod \"79c624de-3c49-430c-b796-d56e850c97d6\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " Jul 08 13:30:26.425531 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425430 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9cfq\" (UniqueName: \"kubernetes.io/projected/79c624de-3c49-430c-b796-d56e850c97d6-kube-api-access-w9cfq\") pod \"79c624de-3c49-430c-b796-d56e850c97d6\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " Jul 08 13:30:26.425584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425529 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-oauth-serving-cert\") pod \"79c624de-3c49-430c-b796-d56e850c97d6\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " Jul 08 13:30:26.425584 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425570 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-service-ca\") pod \"79c624de-3c49-430c-b796-d56e850c97d6\" (UID: \"79c624de-3c49-430c-b796-d56e850c97d6\") " Jul 08 13:30:26.426134 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425659 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-console-config" (OuterVolumeSpecName: "console-config") pod "79c624de-3c49-430c-b796-d56e850c97d6" (UID: "79c624de-3c49-430c-b796-d56e850c97d6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:30:26.426134 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425684 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "79c624de-3c49-430c-b796-d56e850c97d6" (UID: "79c624de-3c49-430c-b796-d56e850c97d6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:30:26.426134 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.425948 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "79c624de-3c49-430c-b796-d56e850c97d6" (UID: "79c624de-3c49-430c-b796-d56e850c97d6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:30:26.426415 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.426138 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-service-ca" (OuterVolumeSpecName: "service-ca") pod "79c624de-3c49-430c-b796-d56e850c97d6" (UID: "79c624de-3c49-430c-b796-d56e850c97d6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:30:26.426415 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.426222 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-oauth-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:30:26.426415 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.426243 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-console-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:30:26.426415 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.426259 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-trusted-ca-bundle\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:30:26.427706 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.427677 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "79c624de-3c49-430c-b796-d56e850c97d6" (UID: "79c624de-3c49-430c-b796-d56e850c97d6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:30:26.427831 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.427721 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "79c624de-3c49-430c-b796-d56e850c97d6" (UID: "79c624de-3c49-430c-b796-d56e850c97d6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:30:26.427831 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.427778 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c624de-3c49-430c-b796-d56e850c97d6-kube-api-access-w9cfq" (OuterVolumeSpecName: "kube-api-access-w9cfq") pod "79c624de-3c49-430c-b796-d56e850c97d6" (UID: "79c624de-3c49-430c-b796-d56e850c97d6"). InnerVolumeSpecName "kube-api-access-w9cfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:30:26.527346 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.527311 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-serving-cert\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:30:26.527346 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.527342 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9cfq\" (UniqueName: \"kubernetes.io/projected/79c624de-3c49-430c-b796-d56e850c97d6-kube-api-access-w9cfq\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:30:26.527346 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.527352 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79c624de-3c49-430c-b796-d56e850c97d6-service-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:30:26.527582 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:26.527363 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79c624de-3c49-430c-b796-d56e850c97d6-console-oauth-config\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:30:27.142015 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.141986 2560 generic.go:356] "Generic (PLEG): container finished" podID="79c624de-3c49-430c-b796-d56e850c97d6" containerID="9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe" exitCode=0 Jul 08 13:30:27.142386 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.142032 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86748f9699-xhmhx" Jul 08 13:30:27.142386 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.142040 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86748f9699-xhmhx" event={"ID":"79c624de-3c49-430c-b796-d56e850c97d6","Type":"ContainerDied","Data":"9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe"} Jul 08 13:30:27.142386 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.142080 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86748f9699-xhmhx" event={"ID":"79c624de-3c49-430c-b796-d56e850c97d6","Type":"ContainerDied","Data":"5be999cc7354290d1a52bc5f74d455ba7710cbe694fca00854ac507cec240f0e"} Jul 08 13:30:27.142386 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.142103 2560 scope.go:117] "RemoveContainer" containerID="9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe" Jul 08 13:30:27.150572 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.150553 2560 scope.go:117] "RemoveContainer" containerID="9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe" Jul 08 13:30:27.150905 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:30:27.150881 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe\": container with ID starting with 9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe not found: ID does not exist" containerID="9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe" Jul 08 13:30:27.150966 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.150937 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe"} err="failed to get container status \"9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe\": rpc error: code = NotFound desc = could not find container \"9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe\": container with ID starting with 9d0a2d2dd3379c23659562370aaf384babf20cc6a4591dd88f653ad661a666fe not found: ID does not exist" Jul 08 13:30:27.158823 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.158795 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86748f9699-xhmhx"] Jul 08 13:30:27.162305 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.162276 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86748f9699-xhmhx"] Jul 08 13:30:27.384078 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.384045 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:27.482872 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.482785 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:27.583075 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:27.583043 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:28.195225 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:28.195178 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:28.250773 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:28.250737 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:29.130817 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:29.130780 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c624de-3c49-430c-b796-d56e850c97d6" path="/var/lib/kubelet/pods/79c624de-3c49-430c-b796-d56e850c97d6/volumes" Jul 08 13:30:43.532981 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:43.532877 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:43.632386 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:43.632332 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:43.996599 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:43.996569 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:55.784391 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:55.784355 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:30:56.022112 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:30:56.022078 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:02.838106 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:02.838072 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:02.949360 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:02.949328 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:03.029376 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:03.029344 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:06.673947 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:06.673905 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:06.775089 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:06.775060 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:39.123209 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:39.123139 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:39.575046 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:39.575014 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:39.660241 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:39.660213 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:39.989005 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:39.988902 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:40.964847 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:40.964794 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:48.569029 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:48.568994 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:50.248957 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:50.248906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:50.351051 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:50.351021 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:56.904714 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.904682 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l"] Jul 08 13:31:56.905086 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.905031 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c624de-3c49-430c-b796-d56e850c97d6" containerName="console" Jul 08 13:31:56.905086 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.905043 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c624de-3c49-430c-b796-d56e850c97d6" containerName="console" Jul 08 13:31:56.905160 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.905098 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c624de-3c49-430c-b796-d56e850c97d6" containerName="console" Jul 08 13:31:56.908129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.908109 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:56.911412 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.911385 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 13:31:56.911546 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.911423 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 13:31:56.911546 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.911397 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 13:31:56.911546 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.911426 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2p2fd\"" Jul 08 13:31:56.917137 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.916452 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l"] Jul 08 13:31:56.968942 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:56.968906 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:31:57.025769 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.025733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.025769 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.025767 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.025986 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.025785 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/918bf1fc-b023-4546-b124-5c28bab91a64-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.025986 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.025801 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddc7n\" (UniqueName: \"kubernetes.io/projected/918bf1fc-b023-4546-b124-5c28bab91a64-kube-api-access-ddc7n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.025986 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.025913 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.026087 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.025985 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.126562 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.126529 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.126740 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.126574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.126740 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.126590 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.126740 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.126705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/918bf1fc-b023-4546-b124-5c28bab91a64-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.126740 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.126734 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddc7n\" (UniqueName: \"kubernetes.io/projected/918bf1fc-b023-4546-b124-5c28bab91a64-kube-api-access-ddc7n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.126992 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.126828 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.126992 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.126948 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.127101 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.127038 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.127659 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.127635 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.129140 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.129121 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/918bf1fc-b023-4546-b124-5c28bab91a64-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.129376 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.129355 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/918bf1fc-b023-4546-b124-5c28bab91a64-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.135410 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.135387 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddc7n\" (UniqueName: \"kubernetes.io/projected/918bf1fc-b023-4546-b124-5c28bab91a64-kube-api-access-ddc7n\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l\" (UID: \"918bf1fc-b023-4546-b124-5c28bab91a64\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.220650 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.220565 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:31:57.353829 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.353805 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l"] Jul 08 13:31:57.355992 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:31:57.355969 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918bf1fc_b023_4546_b124_5c28bab91a64.slice/crio-cc4bc23452e8271b13925698e71597fc28680940f11c207f2d4dfc4c021655f2 WatchSource:0}: Error finding container cc4bc23452e8271b13925698e71597fc28680940f11c207f2d4dfc4c021655f2: Status 404 returned error can't find the container with id cc4bc23452e8271b13925698e71597fc28680940f11c207f2d4dfc4c021655f2 Jul 08 13:31:57.546620 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:57.546530 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" event={"ID":"918bf1fc-b023-4546-b124-5c28bab91a64","Type":"ContainerStarted","Data":"cc4bc23452e8271b13925698e71597fc28680940f11c207f2d4dfc4c021655f2"} Jul 08 13:31:59.556678 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:59.556634 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" event={"ID":"918bf1fc-b023-4546-b124-5c28bab91a64","Type":"ContainerStarted","Data":"faa972f07d8ec843906d72d24057ba749b7c281285ac13a532e38d11cd759dde"} Jul 08 13:31:59.574481 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:31:59.574425 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" podStartSLOduration=2.374426229 podStartE2EDuration="3.574408928s" totalImagesPullingTime="1.199982699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:31:59.572961943 +0000 UTC m=+523.018149614" watchObservedRunningTime="2026-07-08 13:31:59.574408928 +0000 UTC m=+523.019596598" Jul 08 13:32:02.973069 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:02.973034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:32:05.014610 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:05.014580 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:32:07.221468 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:07.221438 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:32:07.221843 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:07.221504 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:32:07.233695 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:07.233665 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:32:07.601655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:07.601625 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l" Jul 08 13:32:09.492950 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:09.492865 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:32:12.396532 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.396499 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-879hk"] Jul 08 13:32:12.399900 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.399881 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.403419 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.403403 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 13:32:12.412442 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.412422 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-879hk"] Jul 08 13:32:12.435760 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.435717 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:32:12.471876 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.471848 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.471995 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.471882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.471995 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.471899 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6rp4\" (UniqueName: \"kubernetes.io/projected/260397f8-b064-44d7-aeda-25f71db304ab-kube-api-access-h6rp4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.472113 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.472006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.472113 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.472041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.472113 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.472075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/260397f8-b064-44d7-aeda-25f71db304ab-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573139 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573301 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573145 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573301 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573162 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6rp4\" (UniqueName: \"kubernetes.io/projected/260397f8-b064-44d7-aeda-25f71db304ab-kube-api-access-h6rp4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573301 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573199 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573301 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573289 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573483 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/260397f8-b064-44d7-aeda-25f71db304ab-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573612 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573591 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573679 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573666 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.573679 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.573670 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.575638 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.575620 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/260397f8-b064-44d7-aeda-25f71db304ab-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.575913 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.575894 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/260397f8-b064-44d7-aeda-25f71db304ab-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.582027 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.582007 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6rp4\" (UniqueName: \"kubernetes.io/projected/260397f8-b064-44d7-aeda-25f71db304ab-kube-api-access-h6rp4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-879hk\" (UID: \"260397f8-b064-44d7-aeda-25f71db304ab\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.709881 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.709804 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:12.844227 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:12.844201 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-879hk"] Jul 08 13:32:12.846850 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:32:12.846820 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod260397f8_b064_44d7_aeda_25f71db304ab.slice/crio-554c8a0eca2076cc9b2bcdcb15098635b2ae1473c3cc1c1367e56f70b1762843 WatchSource:0}: Error finding container 554c8a0eca2076cc9b2bcdcb15098635b2ae1473c3cc1c1367e56f70b1762843: Status 404 returned error can't find the container with id 554c8a0eca2076cc9b2bcdcb15098635b2ae1473c3cc1c1367e56f70b1762843 Jul 08 13:32:13.618762 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:13.618720 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" event={"ID":"260397f8-b064-44d7-aeda-25f71db304ab","Type":"ContainerStarted","Data":"8582fbe53ba48a4fad53a9c36d9525e383749c02eca248e8e5056edfd13fc0ff"} Jul 08 13:32:13.619248 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:13.618774 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" event={"ID":"260397f8-b064-44d7-aeda-25f71db304ab","Type":"ContainerStarted","Data":"554c8a0eca2076cc9b2bcdcb15098635b2ae1473c3cc1c1367e56f70b1762843"} Jul 08 13:32:13.635256 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:13.635204 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" podStartSLOduration=1.420728992 podStartE2EDuration="1.635183509s" totalImagesPullingTime="214.454517ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:32:13.634677831 +0000 UTC m=+537.079865503" watchObservedRunningTime="2026-07-08 13:32:13.635183509 +0000 UTC m=+537.080371182" Jul 08 13:32:15.445172 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:15.445132 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:32:21.487341 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:21.487298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:32:22.710160 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:22.710130 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:22.710553 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:22.710185 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:22.722842 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:22.722816 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:23.684451 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:23.684425 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-879hk" Jul 08 13:32:36.986333 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:36.986304 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf"] Jul 08 13:32:36.991418 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:36.991395 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:36.993981 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:36.993961 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 13:32:37.001154 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.001128 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf"] Jul 08 13:32:37.018748 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.018722 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:32:37.090218 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.090190 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.090358 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.090228 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.090358 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.090251 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f89731ec-5de6-4b5a-855f-780cbda65e8c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.090442 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.090372 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.090442 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.090406 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p8l8\" (UniqueName: \"kubernetes.io/projected/f89731ec-5de6-4b5a-855f-780cbda65e8c-kube-api-access-5p8l8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.090442 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.090428 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.191751 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.191716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p8l8\" (UniqueName: \"kubernetes.io/projected/f89731ec-5de6-4b5a-855f-780cbda65e8c-kube-api-access-5p8l8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.191751 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.191754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.192034 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.191818 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.192034 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.191862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.192034 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.191888 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f89731ec-5de6-4b5a-855f-780cbda65e8c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.192034 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.192022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.192242 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.192222 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.192417 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.192399 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.192466 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.192400 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.194471 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.194452 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f89731ec-5de6-4b5a-855f-780cbda65e8c-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.194620 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.194603 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f89731ec-5de6-4b5a-855f-780cbda65e8c-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.200472 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.200446 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p8l8\" (UniqueName: \"kubernetes.io/projected/f89731ec-5de6-4b5a-855f-780cbda65e8c-kube-api-access-5p8l8\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf\" (UID: \"f89731ec-5de6-4b5a-855f-780cbda65e8c\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.301185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.301105 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:37.436800 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.436770 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf"] Jul 08 13:32:37.439665 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:32:37.439628 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf89731ec_5de6_4b5a_855f_780cbda65e8c.slice/crio-ee7e3a8ce05710e13fc003bd8adb5f32f20021ecc60249070190d4dca8f7fcd5 WatchSource:0}: Error finding container ee7e3a8ce05710e13fc003bd8adb5f32f20021ecc60249070190d4dca8f7fcd5: Status 404 returned error can't find the container with id ee7e3a8ce05710e13fc003bd8adb5f32f20021ecc60249070190d4dca8f7fcd5 Jul 08 13:32:37.743888 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:37.743853 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" event={"ID":"f89731ec-5de6-4b5a-855f-780cbda65e8c","Type":"ContainerStarted","Data":"ee7e3a8ce05710e13fc003bd8adb5f32f20021ecc60249070190d4dca8f7fcd5"} Jul 08 13:32:38.749973 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:38.749916 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" event={"ID":"f89731ec-5de6-4b5a-855f-780cbda65e8c","Type":"ContainerStarted","Data":"630965bf4f92c7cde8f2baba593403ca834bd6dab089d0cca8126d2004eee785"} Jul 08 13:32:38.769533 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:38.769477 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" podStartSLOduration=2.413201953 podStartE2EDuration="2.769462772s" totalImagesPullingTime="356.260819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:32:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:32:38.768181064 +0000 UTC m=+562.213368737" watchObservedRunningTime="2026-07-08 13:32:38.769462772 +0000 UTC m=+562.214650443" Jul 08 13:32:47.302020 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:47.301981 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:47.302496 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:47.302039 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:47.314486 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:47.314460 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:32:47.801948 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:32:47.801903 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf" Jul 08 13:33:16.962647 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:33:16.962617 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:33:16.963615 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:33:16.963594 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:33:21.884133 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:33:21.884095 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:33:23.232208 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:33:23.232170 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:33:56.334225 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:33:56.334153 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:33:56.471303 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:33:56.471260 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:33:56.532112 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:33:56.532084 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:34:01.079185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:34:01.079147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:34:01.179610 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:34:01.179576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:34:02.163015 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:34:02.162980 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:34:02.261606 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:34:02.261577 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:35:33.151409 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:35:33.151312 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:35:34.664790 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:35:34.664750 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:14.439830 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:14.439793 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:14.536612 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:14.536579 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:14.657112 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:14.657076 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:20.238468 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:20.238435 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:20.295654 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:20.295618 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:20.371344 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:20.371309 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:20.470820 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:20.470783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:20.595068 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:20.595034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:26.236214 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:26.236172 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:36:26.336491 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:36:26.336460 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:37:26.108614 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:37:26.108534 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:37:26.159815 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:37:26.159782 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:38:13.967963 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:38:13.967903 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:38:16.992737 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:38:16.992703 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:38:16.994829 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:38:16.994805 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:38:44.425882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:38:44.425844 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:40:07.773220 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:07.773181 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:40:07.876780 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:07.876729 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:40:46.594356 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:46.594321 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9"] Jul 08 13:40:46.594841 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:46.594555 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" podUID="6a46f135-c9aa-4c6d-ae96-009046e88777" containerName="manager" containerID="cri-o://90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948" gracePeriod=10 Jul 08 13:40:50.638670 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.638644 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:40:50.769415 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.769335 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a46f135-c9aa-4c6d-ae96-009046e88777-extensions-socket-volume\") pod \"6a46f135-c9aa-4c6d-ae96-009046e88777\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " Jul 08 13:40:50.769415 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.769397 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8nvq\" (UniqueName: \"kubernetes.io/projected/6a46f135-c9aa-4c6d-ae96-009046e88777-kube-api-access-n8nvq\") pod \"6a46f135-c9aa-4c6d-ae96-009046e88777\" (UID: \"6a46f135-c9aa-4c6d-ae96-009046e88777\") " Jul 08 13:40:50.769788 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.769760 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a46f135-c9aa-4c6d-ae96-009046e88777-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6a46f135-c9aa-4c6d-ae96-009046e88777" (UID: "6a46f135-c9aa-4c6d-ae96-009046e88777"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:40:50.771700 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.771669 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a46f135-c9aa-4c6d-ae96-009046e88777-kube-api-access-n8nvq" (OuterVolumeSpecName: "kube-api-access-n8nvq") pod "6a46f135-c9aa-4c6d-ae96-009046e88777" (UID: "6a46f135-c9aa-4c6d-ae96-009046e88777"). InnerVolumeSpecName "kube-api-access-n8nvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:40:50.870147 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.870119 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6a46f135-c9aa-4c6d-ae96-009046e88777-extensions-socket-volume\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:40:50.870147 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.870144 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8nvq\" (UniqueName: \"kubernetes.io/projected/6a46f135-c9aa-4c6d-ae96-009046e88777-kube-api-access-n8nvq\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:40:50.990317 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.990281 2560 generic.go:356] "Generic (PLEG): container finished" podID="6a46f135-c9aa-4c6d-ae96-009046e88777" containerID="90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948" exitCode=0 Jul 08 13:40:50.990504 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.990339 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" event={"ID":"6a46f135-c9aa-4c6d-ae96-009046e88777","Type":"ContainerDied","Data":"90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948"} Jul 08 13:40:50.990504 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.990369 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" Jul 08 13:40:50.990504 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.990418 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9" event={"ID":"6a46f135-c9aa-4c6d-ae96-009046e88777","Type":"ContainerDied","Data":"8f631e90c0631a1a0ff97a2fda3ae3ca23f6711fc7b0883d71fcce51e314b076"} Jul 08 13:40:50.990504 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:50.990449 2560 scope.go:117] "RemoveContainer" containerID="90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948" Jul 08 13:40:51.000021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:51.000000 2560 scope.go:117] "RemoveContainer" containerID="90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948" Jul 08 13:40:51.000294 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:40:51.000275 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948\": container with ID starting with 90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948 not found: ID does not exist" containerID="90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948" Jul 08 13:40:51.000360 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:51.000302 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948"} err="failed to get container status \"90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948\": rpc error: code = NotFound desc = could not find container \"90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948\": container with ID starting with 90f9d302bcdaed61aa7886deb366cc48b2a1369f1f745a4f422f4c0646b7b948 not found: ID does not exist" Jul 08 13:40:51.033065 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:51.033006 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9"] Jul 08 13:40:51.044251 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:51.044229 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k6kj9"] Jul 08 13:40:51.130692 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:40:51.130663 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a46f135-c9aa-4c6d-ae96-009046e88777" path="/var/lib/kubelet/pods/6a46f135-c9aa-4c6d-ae96-009046e88777/volumes" Jul 08 13:41:47.720900 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.720867 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9"] Jul 08 13:41:47.721373 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.721268 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a46f135-c9aa-4c6d-ae96-009046e88777" containerName="manager" Jul 08 13:41:47.721373 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.721280 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a46f135-c9aa-4c6d-ae96-009046e88777" containerName="manager" Jul 08 13:41:47.721373 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.721351 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a46f135-c9aa-4c6d-ae96-009046e88777" containerName="manager" Jul 08 13:41:47.724228 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.724213 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:47.727098 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.727077 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h4s79\"" Jul 08 13:41:47.735702 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.735680 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9"] Jul 08 13:41:47.856001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.855962 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c49a05b9-3d26-4664-be69-559acbbf4dd2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sb9l9\" (UID: \"c49a05b9-3d26-4664-be69-559acbbf4dd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:47.856001 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.855999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55q6l\" (UniqueName: \"kubernetes.io/projected/c49a05b9-3d26-4664-be69-559acbbf4dd2-kube-api-access-55q6l\") pod \"kuadrant-operator-controller-manager-86878c769f-sb9l9\" (UID: \"c49a05b9-3d26-4664-be69-559acbbf4dd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:47.957056 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.957021 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c49a05b9-3d26-4664-be69-559acbbf4dd2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sb9l9\" (UID: \"c49a05b9-3d26-4664-be69-559acbbf4dd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:47.957056 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.957059 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55q6l\" (UniqueName: \"kubernetes.io/projected/c49a05b9-3d26-4664-be69-559acbbf4dd2-kube-api-access-55q6l\") pod \"kuadrant-operator-controller-manager-86878c769f-sb9l9\" (UID: \"c49a05b9-3d26-4664-be69-559acbbf4dd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:47.957405 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.957388 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c49a05b9-3d26-4664-be69-559acbbf4dd2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-sb9l9\" (UID: \"c49a05b9-3d26-4664-be69-559acbbf4dd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:47.984517 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:47.984454 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55q6l\" (UniqueName: \"kubernetes.io/projected/c49a05b9-3d26-4664-be69-559acbbf4dd2-kube-api-access-55q6l\") pod \"kuadrant-operator-controller-manager-86878c769f-sb9l9\" (UID: \"c49a05b9-3d26-4664-be69-559acbbf4dd2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:48.033852 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:48.033824 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:48.173394 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:48.168592 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9"] Jul 08 13:41:48.245317 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:48.245257 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" event={"ID":"c49a05b9-3d26-4664-be69-559acbbf4dd2","Type":"ContainerStarted","Data":"8b80e866b42d920af1da5bc81b684fa30901546c6967c418af2428783198214f"} Jul 08 13:41:48.245428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:48.245323 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" event={"ID":"c49a05b9-3d26-4664-be69-559acbbf4dd2","Type":"ContainerStarted","Data":"dde9d43cf424f2247c87f66b707bdd063458630a41456fb991b11f8af8e65e93"} Jul 08 13:41:48.245428 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:48.245384 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:41:48.270546 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:48.270500 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" podStartSLOduration=1.270485272 podStartE2EDuration="1.270485272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:41:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:41:48.269813227 +0000 UTC m=+1111.715000899" watchObservedRunningTime="2026-07-08 13:41:48.270485272 +0000 UTC m=+1111.715672942" Jul 08 13:41:59.251911 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:41:59.251881 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-sb9l9" Jul 08 13:43:17.019043 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:43:17.019011 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:43:17.022783 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:43:17.022762 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:43:58.379367 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:43:58.379334 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:44:36.133300 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:44:36.133215 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:45:37.177352 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:45:37.177317 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:37.185344 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:37.185263 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:37.234844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:37.234789 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:42.304180 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:42.304147 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:42.372955 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:42.372908 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:42.522590 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:42.522562 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:42.621160 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:42.621081 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:43.599749 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:43.599721 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:43.702251 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:43.702216 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:43.874631 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:43.874545 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:44.037377 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:44.037343 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:44.304440 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:44.304414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:44.411822 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:44.411792 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:46.813155 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:46.813123 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:48.708458 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:48.708423 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:48.955542 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:48.955511 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:49.064091 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:49.064047 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:49.201506 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:49.201465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:50.675185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:50.675153 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:54.824831 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:54.824796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:54.906700 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:54.906661 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:55.030870 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:55.030836 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:55.104555 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:55.104488 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:56.145404 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:56.145370 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:56.234042 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:56.234004 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:56.297141 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:56.297114 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:56.709061 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:56.709030 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:56.809246 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:56.809198 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:47:59.074730 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:47:59.074699 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:01.286599 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:01.286556 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:01.741251 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:01.741219 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:07.399243 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:07.399207 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:07.500862 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:07.500827 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:09.784839 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:09.784807 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:13.148743 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:13.148708 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:13.208655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:13.208615 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:13.352954 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:13.352904 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:13.454206 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:13.454118 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:14.457630 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:14.457564 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:14.551434 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:14.551399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:14.622060 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:14.622031 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:14.890636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:14.890602 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:15.147115 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:15.147017 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:15.248942 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:15.248884 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:16.588943 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:16.588887 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:17.049051 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:17.048972 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:48:17.055830 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:17.055805 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:48:18.897516 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:18.897482 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:19.097765 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:19.097730 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:19.209581 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:19.209488 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:19.299555 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:19.299511 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:20.623380 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:20.623345 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:25.158438 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:25.158400 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:25.239031 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:25.239002 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:25.631070 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:25.631036 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:26.058534 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:26.058502 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:26.158093 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:26.158015 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:27.537203 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:27.537169 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:32.892551 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:32.892518 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:33.001546 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:33.001515 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:33.093654 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:33.093616 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:33.195289 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:33.195200 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:33.629439 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:33.629401 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:48:59.583091 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:48:59.583055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:05.185550 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:05.185516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:05.239716 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:05.239651 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:05.324097 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:05.324065 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:05.423592 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:05.423554 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:21.457814 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:21.457775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:27.127061 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:27.127001 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:27.232008 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:27.231964 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:27.326829 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:27.326801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:37.844383 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:37.844354 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:38.655895 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:38.655838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:44.663340 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:44.663276 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:44.763006 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:44.762976 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:45.464095 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:45.464055 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:45.556879 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:45.556837 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:45.659152 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:45.659122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:47.255077 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:47.255046 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:52.766381 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:52.766348 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:52.866880 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:52.866846 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:52.966723 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:52.966684 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:53.591556 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:53.591523 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:59.527871 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:59.527817 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:59.589616 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:59.589576 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:49:59.767093 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:49:59.767047 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:00.082197 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:00.082152 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:00.398568 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:00.398437 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:04.363448 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:04.363414 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:04.444737 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:04.444700 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:04.519346 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:04.519309 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:04.901091 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:04.901035 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:05.031692 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:05.031653 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:05.125342 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:05.125298 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:08.869360 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:08.869271 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:09.568621 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:09.568590 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:09.667860 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:09.667828 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:09.994661 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:09.994574 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:15.651095 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:15.651049 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:15.751765 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:15.751719 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:16.121660 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:16.121624 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:16.405253 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:16.405160 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:16.504402 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:16.504366 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:22.186701 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:22.186657 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:22.458095 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:22.458006 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:22.514585 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:22.514545 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:22.890291 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:22.890248 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:23.045160 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:23.045113 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:23.144768 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:23.144662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:29.283312 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:29.283274 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:29.399058 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:29.399029 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:29.517894 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:29.517853 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:29.645151 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:29.645118 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:35.466685 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:35.466643 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:35.537161 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:35.537119 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:42.348875 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:42.348838 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:42.748346 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:42.748249 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:43.031315 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:43.031218 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:45.676061 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:45.676025 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:45.820996 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:45.820946 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:45.934177 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:45.934078 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:46.156695 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:46.156665 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:46.665185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:46.665149 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:46.757490 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:46.757453 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:50.176892 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:50.176853 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:53.856162 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:53.856127 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:56.880182 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:56.880143 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:50:56.980704 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:50:56.980669 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:07.429128 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:07.429097 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:07.526289 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:07.526248 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:12.463950 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:12.463888 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:12.876829 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:12.876796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:18.574464 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:18.574436 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:23.649710 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:23.649677 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:50.718359 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:50.718285 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:51:59.793677 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:51:59.793630 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:00.978298 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:00.978265 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:06.497776 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:06.497738 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:12.376210 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:12.376180 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:12.449164 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:12.449135 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:12.727950 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:12.727833 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:14.005567 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:14.005532 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:14.098156 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:14.098122 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:14.199420 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:14.199379 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:14.946974 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:14.946919 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:15.042018 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:15.041972 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:15.142468 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:15.142433 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:16.481708 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:16.481672 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:22.522873 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:22.522839 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:22.606851 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:22.606821 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:23.187108 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:23.187075 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:29.352758 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:29.352718 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:29.467456 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:29.467407 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:29.556422 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:29.556389 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:40.065364 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:40.065321 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:52:40.428648 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:52:40.428561 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:01.428005 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:01.427969 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:07.227838 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:07.227796 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:07.285190 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:07.285163 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:07.553277 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:07.553228 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:07.874970 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:07.874872 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:07.972691 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:07.972654 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:11.445332 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:11.445284 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:11.614259 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:11.614226 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:14.823167 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:14.823135 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:15.173988 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:15.173899 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:15.503539 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:15.503443 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:15.786843 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:15.786711 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:17.078918 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:17.078886 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:53:17.086236 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:17.086211 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:53:18.466260 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:18.466221 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:18.553689 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:18.553651 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:18.625557 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:18.625521 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:18.725577 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:18.725489 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:19.111041 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:19.111004 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:19.212506 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:19.212465 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:22.851721 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:22.851681 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:29.184611 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.184573 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd"] Jul 08 13:53:29.188144 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.188124 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.190633 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.190608 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ac32a0\"/\"openshift-service-ca.crt\"" Jul 08 13:53:29.191638 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.191615 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-ac32a0\"/\"kube-root-ca.crt\"" Jul 08 13:53:29.191762 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.191615 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ac32a0\"/\"test-model-ac32a0-kserve-self-signed-certs\"" Jul 08 13:53:29.191829 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.191767 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-ac32a0\"/\"default-dockercfg-zx66l\"" Jul 08 13:53:29.197235 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.197213 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd"] Jul 08 13:53:29.226460 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.226429 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:29.313003 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.312963 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73b0a60-db2a-48e6-93b1-17b76f32952a-tls-certs\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.313003 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.313006 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqtwt\" (UniqueName: \"kubernetes.io/projected/d73b0a60-db2a-48e6-93b1-17b76f32952a-kube-api-access-hqtwt\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.313252 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.313065 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-dshm\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.313252 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.313112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-model-cache\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.313252 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.313162 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-tmp-dir\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.313360 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.313256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-home\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414297 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414259 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-tmp-dir\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-home\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73b0a60-db2a-48e6-93b1-17b76f32952a-tls-certs\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqtwt\" (UniqueName: \"kubernetes.io/projected/d73b0a60-db2a-48e6-93b1-17b76f32952a-kube-api-access-hqtwt\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-dshm\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414482 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-model-cache\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414792 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414736 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-tmp-dir\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414792 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414752 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-home\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.414898 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.414843 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-model-cache\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.416853 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.416829 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-dshm\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.417051 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.417016 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73b0a60-db2a-48e6-93b1-17b76f32952a-tls-certs\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.422671 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.422645 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqtwt\" (UniqueName: \"kubernetes.io/projected/d73b0a60-db2a-48e6-93b1-17b76f32952a-kube-api-access-hqtwt\") pod \"test-model-ac32a0-kserve-6df5759b4f-xqdkd\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.499533 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.499439 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:29.645860 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:29.645827 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd"] Jul 08 13:53:29.647761 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:53:29.647733 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd73b0a60_db2a_48e6_93b1_17b76f32952a.slice/crio-2c3fb66253d5d6223fbb02e61ad9e76befd1adc1700887a758a8d8d34b946dd6 WatchSource:0}: Error finding container 2c3fb66253d5d6223fbb02e61ad9e76befd1adc1700887a758a8d8d34b946dd6: Status 404 returned error can't find the container with id 2c3fb66253d5d6223fbb02e61ad9e76befd1adc1700887a758a8d8d34b946dd6 Jul 08 13:53:30.464522 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:30.464483 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" event={"ID":"d73b0a60-db2a-48e6-93b1-17b76f32952a","Type":"ContainerStarted","Data":"215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6"} Jul 08 13:53:30.464522 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:30.464523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" event={"ID":"d73b0a60-db2a-48e6-93b1-17b76f32952a","Type":"ContainerStarted","Data":"2c3fb66253d5d6223fbb02e61ad9e76befd1adc1700887a758a8d8d34b946dd6"} Jul 08 13:53:30.481334 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:30.481280 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" podStartSLOduration=1.481260819 podStartE2EDuration="1.481260819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:53:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:53:30.480135608 +0000 UTC m=+1813.925323278" watchObservedRunningTime="2026-07-08 13:53:30.481260819 +0000 UTC m=+1813.926448488" Jul 08 13:53:31.612296 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:31.612259 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:31.713293 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:31.713254 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:39.499991 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:39.499905 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:39.499991 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:39.500005 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:39.513725 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:39.513695 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:39.525197 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:39.525175 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:53:39.910158 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:39.910124 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:40.010820 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:40.010775 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:51.898520 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.898480 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c"] Jul 08 13:53:51.901892 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.901871 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:51.904435 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.904395 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8f6d76\"/\"kube-root-ca.crt\"" Jul 08 13:53:51.905457 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.905418 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8f6d76\"/\"openshift-service-ca.crt\"" Jul 08 13:53:51.905457 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.905418 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8f6d76\"/\"test-model-8f6d76-kserve-self-signed-certs\"" Jul 08 13:53:51.905662 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.905560 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8f6d76\"/\"default-dockercfg-gtjrl\"" Jul 08 13:53:51.910603 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.910232 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c"] Jul 08 13:53:51.921705 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.921668 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-model-cache\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:51.921859 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.921757 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-dshm\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:51.921859 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.921828 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tls-certs\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:51.921967 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.921880 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tmp-dir\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:51.922012 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.921965 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-home\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:51.922012 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.921989 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92qxg\" (UniqueName: \"kubernetes.io/projected/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-kube-api-access-92qxg\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:51.930100 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:51.930063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:53:52.023452 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-model-cache\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.023655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-dshm\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.023655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023510 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tls-certs\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.023655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023552 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tmp-dir\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.023655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023590 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-home\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.023655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023613 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92qxg\" (UniqueName: \"kubernetes.io/projected/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-kube-api-access-92qxg\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.023905 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023860 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-model-cache\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.024000 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.023961 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-home\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.024042 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.024012 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tmp-dir\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.025963 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.025942 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-dshm\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.026238 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.026218 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tls-certs\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.031601 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.031560 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92qxg\" (UniqueName: \"kubernetes.io/projected/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-kube-api-access-92qxg\") pod \"test-model-8f6d76-kserve-6668844d8c-fgp2c\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.214220 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.214112 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:53:52.358804 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.358729 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c"] Jul 08 13:53:52.361362 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:53:52.361335 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5bc4f41_0d63_413f_8f78_8d6fbdee645f.slice/crio-8d01a4815b99f7ae011ad8456228c0a1932ca21daaa81c18a06a5d9a4f1a8f8e WatchSource:0}: Error finding container 8d01a4815b99f7ae011ad8456228c0a1932ca21daaa81c18a06a5d9a4f1a8f8e: Status 404 returned error can't find the container with id 8d01a4815b99f7ae011ad8456228c0a1932ca21daaa81c18a06a5d9a4f1a8f8e Jul 08 13:53:52.575511 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.575477 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" event={"ID":"b5bc4f41-0d63-413f-8f78-8d6fbdee645f","Type":"ContainerStarted","Data":"ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64"} Jul 08 13:53:52.575690 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.575523 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" event={"ID":"b5bc4f41-0d63-413f-8f78-8d6fbdee645f","Type":"ContainerStarted","Data":"8d01a4815b99f7ae011ad8456228c0a1932ca21daaa81c18a06a5d9a4f1a8f8e"} Jul 08 13:53:52.592675 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:53:52.592612 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" podStartSLOduration=1.592595632 podStartE2EDuration="1.592595632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:53:52.59157199 +0000 UTC m=+1836.036759659" watchObservedRunningTime="2026-07-08 13:53:52.592595632 +0000 UTC m=+1836.037783315" Jul 08 13:54:02.214417 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:02.214359 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:54:02.216982 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:02.214438 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:54:02.227645 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:02.227621 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:54:02.633260 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:02.633218 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:54:41.899129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:41.899047 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd"] Jul 08 13:54:41.923173 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:41.923131 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:54:42.871516 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:42.871478 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c"] Jul 08 13:54:43.151823 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:43.151712 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:54:43.486087 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:43.485950 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:54:43.588140 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:43.588085 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:54:43.882574 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:43.882517 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:54:43.981648 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:43.981608 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:54:48.539738 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:48.539700 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:54:56.921184 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:56.921141 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" podUID="d73b0a60-db2a-48e6-93b1-17b76f32952a" containerName="main" containerID="cri-o://215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6" gracePeriod=45 Jul 08 13:54:57.179196 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.179127 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:54:57.261770 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.261732 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73b0a60-db2a-48e6-93b1-17b76f32952a-tls-certs\") pod \"d73b0a60-db2a-48e6-93b1-17b76f32952a\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " Jul 08 13:54:57.262010 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.261798 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-model-cache\") pod \"d73b0a60-db2a-48e6-93b1-17b76f32952a\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " Jul 08 13:54:57.262010 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.261819 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-home\") pod \"d73b0a60-db2a-48e6-93b1-17b76f32952a\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " Jul 08 13:54:57.262010 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.261842 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqtwt\" (UniqueName: \"kubernetes.io/projected/d73b0a60-db2a-48e6-93b1-17b76f32952a-kube-api-access-hqtwt\") pod \"d73b0a60-db2a-48e6-93b1-17b76f32952a\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " Jul 08 13:54:57.262010 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.261886 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-tmp-dir\") pod \"d73b0a60-db2a-48e6-93b1-17b76f32952a\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " Jul 08 13:54:57.262010 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.261913 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-dshm\") pod \"d73b0a60-db2a-48e6-93b1-17b76f32952a\" (UID: \"d73b0a60-db2a-48e6-93b1-17b76f32952a\") " Jul 08 13:54:57.262691 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.262646 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-home" (OuterVolumeSpecName: "home") pod "d73b0a60-db2a-48e6-93b1-17b76f32952a" (UID: "d73b0a60-db2a-48e6-93b1-17b76f32952a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:57.262832 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.262769 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-model-cache" (OuterVolumeSpecName: "model-cache") pod "d73b0a60-db2a-48e6-93b1-17b76f32952a" (UID: "d73b0a60-db2a-48e6-93b1-17b76f32952a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:57.262832 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.262811 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d73b0a60-db2a-48e6-93b1-17b76f32952a" (UID: "d73b0a60-db2a-48e6-93b1-17b76f32952a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:57.264347 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.264325 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-dshm" (OuterVolumeSpecName: "dshm") pod "d73b0a60-db2a-48e6-93b1-17b76f32952a" (UID: "d73b0a60-db2a-48e6-93b1-17b76f32952a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:57.264649 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.264628 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d73b0a60-db2a-48e6-93b1-17b76f32952a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d73b0a60-db2a-48e6-93b1-17b76f32952a" (UID: "d73b0a60-db2a-48e6-93b1-17b76f32952a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:54:57.264732 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.264628 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d73b0a60-db2a-48e6-93b1-17b76f32952a-kube-api-access-hqtwt" (OuterVolumeSpecName: "kube-api-access-hqtwt") pod "d73b0a60-db2a-48e6-93b1-17b76f32952a" (UID: "d73b0a60-db2a-48e6-93b1-17b76f32952a"). InnerVolumeSpecName "kube-api-access-hqtwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:54:57.363049 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.363016 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-model-cache\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:57.363049 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.363044 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-home\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:57.363049 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.363054 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqtwt\" (UniqueName: \"kubernetes.io/projected/d73b0a60-db2a-48e6-93b1-17b76f32952a-kube-api-access-hqtwt\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:57.363307 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.363065 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-tmp-dir\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:57.363307 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.363077 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d73b0a60-db2a-48e6-93b1-17b76f32952a-dshm\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:57.363307 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.363084 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d73b0a60-db2a-48e6-93b1-17b76f32952a-tls-certs\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:57.890085 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.890049 2560 generic.go:356] "Generic (PLEG): container finished" podID="d73b0a60-db2a-48e6-93b1-17b76f32952a" containerID="215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6" exitCode=0 Jul 08 13:54:57.890282 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.890095 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" Jul 08 13:54:57.890282 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.890124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" event={"ID":"d73b0a60-db2a-48e6-93b1-17b76f32952a","Type":"ContainerDied","Data":"215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6"} Jul 08 13:54:57.890282 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.890168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd" event={"ID":"d73b0a60-db2a-48e6-93b1-17b76f32952a","Type":"ContainerDied","Data":"2c3fb66253d5d6223fbb02e61ad9e76befd1adc1700887a758a8d8d34b946dd6"} Jul 08 13:54:57.890282 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.890187 2560 scope.go:117] "RemoveContainer" containerID="215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6" Jul 08 13:54:57.892970 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.892914 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" podUID="b5bc4f41-0d63-413f-8f78-8d6fbdee645f" containerName="main" containerID="cri-o://ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64" gracePeriod=45 Jul 08 13:54:57.901179 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.901150 2560 scope.go:117] "RemoveContainer" containerID="215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6" Jul 08 13:54:57.901490 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:54:57.901460 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6\": container with ID starting with 215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6 not found: ID does not exist" containerID="215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6" Jul 08 13:54:57.901537 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.901500 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6"} err="failed to get container status \"215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6\": rpc error: code = NotFound desc = could not find container \"215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6\": container with ID starting with 215e8a7637107340a9628c06948233c0404e87eeee91525bbf101aa626272cc6 not found: ID does not exist" Jul 08 13:54:57.913179 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.913144 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd"] Jul 08 13:54:57.916323 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:57.916290 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-ac32a0/test-model-ac32a0-kserve-6df5759b4f-xqdkd"] Jul 08 13:54:58.149300 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.149235 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:54:58.271319 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271281 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-dshm\") pod \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " Jul 08 13:54:58.271513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271334 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-model-cache\") pod \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " Jul 08 13:54:58.271513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271357 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tls-certs\") pod \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " Jul 08 13:54:58.271513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271386 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92qxg\" (UniqueName: \"kubernetes.io/projected/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-kube-api-access-92qxg\") pod \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " Jul 08 13:54:58.271513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271410 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tmp-dir\") pod \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " Jul 08 13:54:58.271513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271438 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-home\") pod \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\" (UID: \"b5bc4f41-0d63-413f-8f78-8d6fbdee645f\") " Jul 08 13:54:58.271786 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271636 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-model-cache" (OuterVolumeSpecName: "model-cache") pod "b5bc4f41-0d63-413f-8f78-8d6fbdee645f" (UID: "b5bc4f41-0d63-413f-8f78-8d6fbdee645f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:58.271786 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271674 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b5bc4f41-0d63-413f-8f78-8d6fbdee645f" (UID: "b5bc4f41-0d63-413f-8f78-8d6fbdee645f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:58.271786 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.271760 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-home" (OuterVolumeSpecName: "home") pod "b5bc4f41-0d63-413f-8f78-8d6fbdee645f" (UID: "b5bc4f41-0d63-413f-8f78-8d6fbdee645f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:58.273813 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.273789 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-dshm" (OuterVolumeSpecName: "dshm") pod "b5bc4f41-0d63-413f-8f78-8d6fbdee645f" (UID: "b5bc4f41-0d63-413f-8f78-8d6fbdee645f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 13:54:58.274135 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.274113 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b5bc4f41-0d63-413f-8f78-8d6fbdee645f" (UID: "b5bc4f41-0d63-413f-8f78-8d6fbdee645f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:54:58.274135 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.274129 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-kube-api-access-92qxg" (OuterVolumeSpecName: "kube-api-access-92qxg") pod "b5bc4f41-0d63-413f-8f78-8d6fbdee645f" (UID: "b5bc4f41-0d63-413f-8f78-8d6fbdee645f"). InnerVolumeSpecName "kube-api-access-92qxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:54:58.374285 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.374243 2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-dshm\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:58.374285 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.374284 2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-model-cache\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:58.374557 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.374304 2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tls-certs\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:58.374557 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.374326 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92qxg\" (UniqueName: \"kubernetes.io/projected/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-kube-api-access-92qxg\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:58.374557 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.374341 2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-tmp-dir\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:58.374557 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.374355 2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5bc4f41-0d63-413f-8f78-8d6fbdee645f-home\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:54:58.896520 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.896481 2560 generic.go:356] "Generic (PLEG): container finished" podID="b5bc4f41-0d63-413f-8f78-8d6fbdee645f" containerID="ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64" exitCode=0 Jul 08 13:54:58.896821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.896542 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" event={"ID":"b5bc4f41-0d63-413f-8f78-8d6fbdee645f","Type":"ContainerDied","Data":"ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64"} Jul 08 13:54:58.896821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.896574 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" event={"ID":"b5bc4f41-0d63-413f-8f78-8d6fbdee645f","Type":"ContainerDied","Data":"8d01a4815b99f7ae011ad8456228c0a1932ca21daaa81c18a06a5d9a4f1a8f8e"} Jul 08 13:54:58.896821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.896592 2560 scope.go:117] "RemoveContainer" containerID="ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64" Jul 08 13:54:58.896821 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.896548 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c" Jul 08 13:54:58.905882 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.905863 2560 scope.go:117] "RemoveContainer" containerID="ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64" Jul 08 13:54:58.906230 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:54:58.906206 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64\": container with ID starting with ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64 not found: ID does not exist" containerID="ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64" Jul 08 13:54:58.906301 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.906241 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64"} err="failed to get container status \"ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64\": rpc error: code = NotFound desc = could not find container \"ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64\": container with ID starting with ab95f08fbf6c58ceb52b9896dea413f4b26825c94a6aae0b7399bec4abea6e64 not found: ID does not exist" Jul 08 13:54:58.917916 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.917828 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c"] Jul 08 13:54:58.919867 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:58.919828 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8f6d76/test-model-8f6d76-kserve-6668844d8c-fgp2c"] Jul 08 13:54:59.131477 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:59.131436 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5bc4f41-0d63-413f-8f78-8d6fbdee645f" path="/var/lib/kubelet/pods/b5bc4f41-0d63-413f-8f78-8d6fbdee645f/volumes" Jul 08 13:54:59.131899 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:54:59.131884 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d73b0a60-db2a-48e6-93b1-17b76f32952a" path="/var/lib/kubelet/pods/d73b0a60-db2a-48e6-93b1-17b76f32952a/volumes" Jul 08 13:55:17.609388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:17.609353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:25.361348 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:25.361313 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:25.649131 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:25.649034 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:26.022293 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:26.022183 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:26.479835 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:26.479801 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:26.581777 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:26.581739 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:26.678362 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:26.678327 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:26.787192 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:26.787094 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:32.968860 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:32.968817 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:33.041189 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:33.041148 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:33.297458 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:33.297361 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:33.623535 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:33.623506 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:33.969209 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:33.969113 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:37.056604 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:37.056571 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:37.176669 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:37.176628 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:37.334449 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:37.334302 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:37.731193 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:37.731067 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:37.839300 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:37.839266 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:37.940511 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:37.940472 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:41.546050 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:41.546009 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:41.995021 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:41.994984 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:42.317228 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:42.317182 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:42.857709 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:42.857660 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:45.206844 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.206757 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:45.280019 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.279985 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9"] Jul 08 13:55:45.280367 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.280355 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d73b0a60-db2a-48e6-93b1-17b76f32952a" containerName="main" Jul 08 13:55:45.280411 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.280369 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d73b0a60-db2a-48e6-93b1-17b76f32952a" containerName="main" Jul 08 13:55:45.280411 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.280377 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5bc4f41-0d63-413f-8f78-8d6fbdee645f" containerName="main" Jul 08 13:55:45.280411 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.280381 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5bc4f41-0d63-413f-8f78-8d6fbdee645f" containerName="main" Jul 08 13:55:45.280506 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.280455 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5bc4f41-0d63-413f-8f78-8d6fbdee645f" containerName="main" Jul 08 13:55:45.280506 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.280465 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d73b0a60-db2a-48e6-93b1-17b76f32952a" containerName="main" Jul 08 13:55:45.283544 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.283524 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.286183 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.286156 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d1efd5-serving-cert\"" Jul 08 13:55:45.286308 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.286267 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hz4c8\"" Jul 08 13:55:45.287171 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.287153 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 13:55:45.287283 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.287195 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 13:55:45.293081 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.292984 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9"] Jul 08 13:55:45.298482 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.298458 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:45.413331 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.413288 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfwv6\" (UniqueName: \"kubernetes.io/projected/40c2a30a-8a43-477c-a669-ff3a0f74fe90-kube-api-access-pfwv6\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.413521 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.413339 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2a30a-8a43-477c-a669-ff3a0f74fe90-openshift-service-ca\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.413521 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.413380 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2a30a-8a43-477c-a669-ff3a0f74fe90-maas-api-tls\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.424301 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.424267 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:45.514972 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.514825 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfwv6\" (UniqueName: \"kubernetes.io/projected/40c2a30a-8a43-477c-a669-ff3a0f74fe90-kube-api-access-pfwv6\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.514972 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.514882 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2a30a-8a43-477c-a669-ff3a0f74fe90-openshift-service-ca\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.514972 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.514918 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2a30a-8a43-477c-a669-ff3a0f74fe90-maas-api-tls\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.515581 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.515553 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2a30a-8a43-477c-a669-ff3a0f74fe90-openshift-service-ca\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.517610 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.517586 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2a30a-8a43-477c-a669-ff3a0f74fe90-maas-api-tls\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.528608 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.528577 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfwv6\" (UniqueName: \"kubernetes.io/projected/40c2a30a-8a43-477c-a669-ff3a0f74fe90-kube-api-access-pfwv6\") pod \"maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.596556 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.596513 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:45.601441 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.601413 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:45.742730 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.742702 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9"] Jul 08 13:55:45.745430 ip-10-0-129-157 kubenswrapper[2560]: W0708 13:55:45.745405 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c2a30a_8a43_477c_a669_ff3a0f74fe90.slice/crio-7c109c530be66c9c5ad6d4a627561c73758a1dca6a693ed00995aaf435557c8d WatchSource:0}: Error finding container 7c109c530be66c9c5ad6d4a627561c73758a1dca6a693ed00995aaf435557c8d: Status 404 returned error can't find the container with id 7c109c530be66c9c5ad6d4a627561c73758a1dca6a693ed00995aaf435557c8d Jul 08 13:55:45.746692 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.746668 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 13:55:45.807994 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.807957 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:45.909055 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:45.909022 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:46.136311 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:46.136219 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" event={"ID":"40c2a30a-8a43-477c-a669-ff3a0f74fe90","Type":"ContainerStarted","Data":"7c109c530be66c9c5ad6d4a627561c73758a1dca6a693ed00995aaf435557c8d"} Jul 08 13:55:49.155376 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:49.155341 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" event={"ID":"40c2a30a-8a43-477c-a669-ff3a0f74fe90","Type":"ContainerStarted","Data":"5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33"} Jul 08 13:55:49.155817 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:49.155404 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:49.174556 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:49.174503 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" podStartSLOduration=1.150869579 podStartE2EDuration="4.174483882s" totalImagesPullingTime="3.023614303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 13:55:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 13:55:49.172765429 +0000 UTC m=+1952.617953103" watchObservedRunningTime="2026-07-08 13:55:49.174483882 +0000 UTC m=+1952.619671552" Jul 08 13:55:49.680351 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:49.680309 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:55.166404 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:55.166373 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:55:57.589022 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:57.588982 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:57.696221 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:57.696184 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:58.688604 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:58.688561 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:55:58.789381 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:55:58.789342 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:03.986704 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:03.986665 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:04.076866 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:04.076824 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:04.261618 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:04.261516 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:04.632250 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:04.632215 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:05.033388 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.033290 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:05.133195 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.133157 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:05.281641 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.281605 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9"] Jul 08 13:56:05.281872 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.281839 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" podUID="40c2a30a-8a43-477c-a669-ff3a0f74fe90" containerName="maas-api" containerID="cri-o://5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33" gracePeriod=30 Jul 08 13:56:05.322041 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.322008 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:05.532497 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.532472 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:56:05.615759 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.615722 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfwv6\" (UniqueName: \"kubernetes.io/projected/40c2a30a-8a43-477c-a669-ff3a0f74fe90-kube-api-access-pfwv6\") pod \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " Jul 08 13:56:05.615993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.615780 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2a30a-8a43-477c-a669-ff3a0f74fe90-openshift-service-ca\") pod \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " Jul 08 13:56:05.615993 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.615882 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2a30a-8a43-477c-a669-ff3a0f74fe90-maas-api-tls\") pod \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\" (UID: \"40c2a30a-8a43-477c-a669-ff3a0f74fe90\") " Jul 08 13:56:05.616254 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.616231 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c2a30a-8a43-477c-a669-ff3a0f74fe90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40c2a30a-8a43-477c-a669-ff3a0f74fe90" (UID: "40c2a30a-8a43-477c-a669-ff3a0f74fe90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 13:56:05.618429 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.618403 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40c2a30a-8a43-477c-a669-ff3a0f74fe90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40c2a30a-8a43-477c-a669-ff3a0f74fe90" (UID: "40c2a30a-8a43-477c-a669-ff3a0f74fe90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 13:56:05.618518 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.618461 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c2a30a-8a43-477c-a669-ff3a0f74fe90-kube-api-access-pfwv6" (OuterVolumeSpecName: "kube-api-access-pfwv6") pod "40c2a30a-8a43-477c-a669-ff3a0f74fe90" (UID: "40c2a30a-8a43-477c-a669-ff3a0f74fe90"). InnerVolumeSpecName "kube-api-access-pfwv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 13:56:05.717238 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.717202 2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40c2a30a-8a43-477c-a669-ff3a0f74fe90-maas-api-tls\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:56:05.717238 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.717236 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfwv6\" (UniqueName: \"kubernetes.io/projected/40c2a30a-8a43-477c-a669-ff3a0f74fe90-kube-api-access-pfwv6\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:56:05.717238 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:05.717246 2560 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40c2a30a-8a43-477c-a669-ff3a0f74fe90-openshift-service-ca\") on node \"ip-10-0-129-157.ec2.internal\" DevicePath \"\"" Jul 08 13:56:06.238358 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.238319 2560 generic.go:356] "Generic (PLEG): container finished" podID="40c2a30a-8a43-477c-a669-ff3a0f74fe90" containerID="5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33" exitCode=0 Jul 08 13:56:06.238358 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.238369 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" Jul 08 13:56:06.238935 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.238400 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" event={"ID":"40c2a30a-8a43-477c-a669-ff3a0f74fe90","Type":"ContainerDied","Data":"5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33"} Jul 08 13:56:06.238935 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.238452 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9" event={"ID":"40c2a30a-8a43-477c-a669-ff3a0f74fe90","Type":"ContainerDied","Data":"7c109c530be66c9c5ad6d4a627561c73758a1dca6a693ed00995aaf435557c8d"} Jul 08 13:56:06.238935 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.238475 2560 scope.go:117] "RemoveContainer" containerID="5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33" Jul 08 13:56:06.248065 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.248045 2560 scope.go:117] "RemoveContainer" containerID="5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33" Jul 08 13:56:06.248334 ip-10-0-129-157 kubenswrapper[2560]: E0708 13:56:06.248314 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33\": container with ID starting with 5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33 not found: ID does not exist" containerID="5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33" Jul 08 13:56:06.248402 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.248344 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33"} err="failed to get container status \"5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33\": rpc error: code = NotFound desc = could not find container \"5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33\": container with ID starting with 5e5dfddf1d9ed1d1878754894f9d714f54011df3f53ec2f9114dce23311cce33 not found: ID does not exist" Jul 08 13:56:06.263171 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.263142 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9"] Jul 08 13:56:06.267430 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:06.267403 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d1efd5-69b88556d4-z5lj9"] Jul 08 13:56:07.130765 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:07.130731 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c2a30a-8a43-477c-a669-ff3a0f74fe90" path="/var/lib/kubelet/pods/40c2a30a-8a43-477c-a669-ff3a0f74fe90/volumes" Jul 08 13:56:11.358438 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:11.358399 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:11.457533 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:11.457493 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:11.610757 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:11.610663 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:12.159396 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:12.159353 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:12.483186 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:12.483104 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:12.694646 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:12.694608 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:15.342433 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:15.342390 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:15.436488 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:15.436427 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:15.524673 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:15.524635 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:15.700634 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:15.700535 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:15.878545 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:15.878506 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:15.979275 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:15.979188 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:19.799749 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:19.799707 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:20.116458 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:20.116423 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:20.431274 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:20.431193 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:20.733812 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:20.733706 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:23.333513 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:23.333463 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:23.435259 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:23.435222 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:23.635824 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:23.635723 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:23.735168 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:23.735130 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:23.976902 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:23.976783 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:24.077695 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:24.077661 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:27.878057 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:27.878012 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:36.636305 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:36.636265 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:36.736164 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:36.736124 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:37.872557 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:37.872518 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:45.541101 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:45.541063 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:56:45.636060 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:56:45.636029 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:57:10.663678 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:57:10.663647 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:57:29.233129 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:57:29.233092 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:57:29.311943 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:57:29.311902 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:58:17.110833 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:58:17.110792 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:58:17.118153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:58:17.118133 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gt94q_f8039f33-da31-4086-bf04-ceb18cc54ac5/ovn-acl-logging/0.log" Jul 08 13:58:48.793573 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:58:48.793521 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:32.834286 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:32.834216 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:33.268636 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:33.268542 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:33.380600 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:33.380567 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:33.582878 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:33.582839 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:39.041271 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:39.041220 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:39.113456 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:39.113422 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:39.316151 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:39.316119 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:39.708179 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:39.708099 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:40.106869 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:40.106836 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:40.206610 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:40.206567 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:40.309185 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:40.309158 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:40.411595 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:40.411501 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:46.117864 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:46.117829 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:47.726589 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:47.726537 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 13:59:51.815655 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:51.815624 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w8rsg_028c363b-2646-4686-b821-4847944bf71d/manager/0.log" Jul 08 13:59:52.309442 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:52.309415 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-6thdj_f2769322-e620-458e-804e-0407f9efea88/copy-manifests/0.log" Jul 08 13:59:52.322156 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:52.322137 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-6thdj_f2769322-e620-458e-804e-0407f9efea88/manager/0.log" Jul 08 13:59:54.256199 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:54.256169 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mmvrg_3a35435b-824e-4f94-b8df-5f45a44d951f/kuadrant-console-plugin/0.log" Jul 08 13:59:54.510153 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:54.510083 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sb9l9_c49a05b9-3d26-4664-be69-559acbbf4dd2/manager/0.log" Jul 08 13:59:55.334741 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:55.334705 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746f76655b-glgm9_1e3da55b-5ba1-4d72-8372-388176e40522/kube-auth-proxy/0.log" Jul 08 13:59:56.262892 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:56.262852 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-mfw5l_918bf1fc-b023-4546-b124-5c28bab91a64/main/0.log" Jul 08 13:59:56.495208 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:56.495178 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-879hk_260397f8-b064-44d7-aeda-25f71db304ab/main/0.log" Jul 08 13:59:56.841508 ip-10-0-129-157 kubenswrapper[2560]: I0708 13:59:56.841477 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-6kjbf_f89731ec-5de6-4b5a-855f-780cbda65e8c/main/0.log" Jul 08 14:00:01.097560 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:01.097517 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 14:00:01.302722 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:01.302662 2560 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" limit=1024 Jul 08 14:00:04.815348 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:04.815316 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ttqmm_9ed8e9d3-dce6-4a00-99aa-f300ece48f4e/global-pull-secret-syncer/0.log" Jul 08 14:00:04.861626 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:04.861601 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-c2m6r_1589525f-a8f4-4b36-8285-1efbd90f864a/konnectivity-agent/0.log" Jul 08 14:00:04.958938 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:04.958899 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-157.ec2.internal_78255babc9a1b932c1400b78f8b89518/haproxy/0.log" Jul 08 14:00:10.235717 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:10.235690 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-mmvrg_3a35435b-824e-4f94-b8df-5f45a44d951f/kuadrant-console-plugin/0.log" Jul 08 14:00:10.386843 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:10.386813 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-sb9l9_c49a05b9-3d26-4664-be69-559acbbf4dd2/manager/0.log" Jul 08 14:00:12.413854 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.413827 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g9nbm_d009ae46-a832-47d9-927f-27117e30087c/kube-state-metrics/0.log" Jul 08 14:00:12.433408 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.433363 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g9nbm_d009ae46-a832-47d9-927f-27117e30087c/kube-rbac-proxy-main/0.log" Jul 08 14:00:12.452610 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.452589 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-g9nbm_d009ae46-a832-47d9-927f-27117e30087c/kube-rbac-proxy-self/0.log" Jul 08 14:00:12.520115 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.520052 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-h87vn_b7f79f42-f363-4af1-9406-006f224c9ee8/monitoring-plugin/0.log" Jul 08 14:00:12.656175 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.656153 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9bg7q_8802fd81-6a37-4b94-9231-526ae61daa5f/node-exporter/0.log" Jul 08 14:00:12.689829 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.689804 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9bg7q_8802fd81-6a37-4b94-9231-526ae61daa5f/kube-rbac-proxy/0.log" Jul 08 14:00:12.713087 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.713067 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9bg7q_8802fd81-6a37-4b94-9231-526ae61daa5f/init-textfile/0.log" Jul 08 14:00:12.907659 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.907624 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f1384ded-9af6-4b93-840c-14ceff895960/prometheus/0.log" Jul 08 14:00:12.931612 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.931589 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f1384ded-9af6-4b93-840c-14ceff895960/config-reloader/0.log" Jul 08 14:00:12.951526 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.951494 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f1384ded-9af6-4b93-840c-14ceff895960/thanos-sidecar/0.log" Jul 08 14:00:12.971836 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.971816 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f1384ded-9af6-4b93-840c-14ceff895960/kube-rbac-proxy-web/0.log" Jul 08 14:00:12.991820 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:12.991801 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f1384ded-9af6-4b93-840c-14ceff895960/kube-rbac-proxy/0.log" Jul 08 14:00:13.013266 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.013238 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f1384ded-9af6-4b93-840c-14ceff895960/kube-rbac-proxy-thanos/0.log" Jul 08 14:00:13.034032 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.034012 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f1384ded-9af6-4b93-840c-14ceff895960/init-config-reloader/0.log" Jul 08 14:00:13.061572 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.061541 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-72fnj_075f34cd-c5b7-4c0e-ade4-e8d7aea29594/prometheus-operator/0.log" Jul 08 14:00:13.080623 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.080597 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-72fnj_075f34cd-c5b7-4c0e-ade4-e8d7aea29594/kube-rbac-proxy/0.log" Jul 08 14:00:13.109344 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.109315 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-562ds_f83a92ef-e215-4f00-b90a-3fedcb6b45b1/prometheus-operator-admission-webhook/0.log" Jul 08 14:00:13.137250 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.137226 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b45c677b7-qmx7j_dca8ce09-0126-4e51-a003-4aece88b7143/telemeter-client/0.log" Jul 08 14:00:13.156042 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.156002 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b45c677b7-qmx7j_dca8ce09-0126-4e51-a003-4aece88b7143/reload/0.log" Jul 08 14:00:13.175763 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.175707 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b45c677b7-qmx7j_dca8ce09-0126-4e51-a003-4aece88b7143/kube-rbac-proxy/0.log" Jul 08 14:00:13.204214 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.204187 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cf74f759f-hmwjj_69f6fd31-303a-40c8-8e37-527986051c22/thanos-query/0.log" Jul 08 14:00:13.223644 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.223626 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cf74f759f-hmwjj_69f6fd31-303a-40c8-8e37-527986051c22/kube-rbac-proxy-web/0.log" Jul 08 14:00:13.244934 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.244908 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cf74f759f-hmwjj_69f6fd31-303a-40c8-8e37-527986051c22/kube-rbac-proxy/0.log" Jul 08 14:00:13.264468 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.264447 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cf74f759f-hmwjj_69f6fd31-303a-40c8-8e37-527986051c22/prom-label-proxy/0.log" Jul 08 14:00:13.294539 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.294524 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cf74f759f-hmwjj_69f6fd31-303a-40c8-8e37-527986051c22/kube-rbac-proxy-rules/0.log" Jul 08 14:00:13.317438 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:13.317423 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6cf74f759f-hmwjj_69f6fd31-303a-40c8-8e37-527986051c22/kube-rbac-proxy-metrics/0.log" Jul 08 14:00:15.657664 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.657629 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm"] Jul 08 14:00:15.658047 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.658027 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c2a30a-8a43-477c-a669-ff3a0f74fe90" containerName="maas-api" Jul 08 14:00:15.658047 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.658038 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c2a30a-8a43-477c-a669-ff3a0f74fe90" containerName="maas-api" Jul 08 14:00:15.658123 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.658093 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c2a30a-8a43-477c-a669-ff3a0f74fe90" containerName="maas-api" Jul 08 14:00:15.661296 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.661276 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.663677 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.663655 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tljhk\"/\"openshift-service-ca.crt\"" Jul 08 14:00:15.663782 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.663662 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tljhk\"/\"kube-root-ca.crt\"" Jul 08 14:00:15.664651 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.664635 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tljhk\"/\"default-dockercfg-kt4cr\"" Jul 08 14:00:15.671679 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.671657 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm"] Jul 08 14:00:15.675589 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.675573 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c97db7bdf-cthh6_d2df0b45-4886-4a96-8bea-6216c3a4c3fa/console/0.log" Jul 08 14:00:15.822277 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.822240 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-podres\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.822433 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.822289 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v75fj\" (UniqueName: \"kubernetes.io/projected/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-kube-api-access-v75fj\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.822433 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.822363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-sys\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.822585 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.822463 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-lib-modules\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.822585 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.822517 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-proc\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923020 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.922917 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-lib-modules\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923020 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.922998 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-proc\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923224 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.923031 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-podres\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923224 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.923068 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v75fj\" (UniqueName: \"kubernetes.io/projected/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-kube-api-access-v75fj\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923224 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.923088 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-lib-modules\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923224 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.923110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-sys\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923224 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.923141 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-sys\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923224 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.923108 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-proc\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.923224 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.923166 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-podres\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.931384 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.931356 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v75fj\" (UniqueName: \"kubernetes.io/projected/d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd-kube-api-access-v75fj\") pod \"perf-node-gather-daemonset-p5npm\" (UID: \"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd\") " pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:15.971017 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:15.970993 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:16.118471 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:16.118428 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm"] Jul 08 14:00:16.384597 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:16.384566 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" event={"ID":"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd","Type":"ContainerStarted","Data":"17caa1207f59b3b5f2208abd6b09dfc258c6c68ff9e5f9be05aeefcdd3cef1b2"} Jul 08 14:00:16.384777 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:16.384616 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" event={"ID":"d5cb56b4-d62c-4d8a-8ae0-75b40cc223cd","Type":"ContainerStarted","Data":"dc97a5e5bf74bf91aa1033502d87e3f99a46fd0f1600d6bdf7a12ec29146b00c"} Jul 08 14:00:16.384777 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:16.384653 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:16.402205 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:16.402158 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" podStartSLOduration=1.402143266 podStartE2EDuration="1.402143266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 14:00:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 14:00:16.400527445 +0000 UTC m=+2219.845715116" watchObservedRunningTime="2026-07-08 14:00:16.402143266 +0000 UTC m=+2219.847330936" Jul 08 14:00:17.437401 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:17.437374 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2lfqh_3400c925-fed7-4390-b1c3-0a84a7a57165/dns/0.log" Jul 08 14:00:17.460842 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:17.460818 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2lfqh_3400c925-fed7-4390-b1c3-0a84a7a57165/kube-rbac-proxy/0.log" Jul 08 14:00:17.592147 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:17.592117 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v2rcg_b42a58a6-b426-475f-ba38-e40d34589fbc/dns-node-resolver/0.log" Jul 08 14:00:18.181765 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:18.181737 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m44xj_26c9c0ad-a236-49e6-b425-7a867b9be873/node-ca/0.log" Jul 08 14:00:19.307831 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:19.307799 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746f76655b-glgm9_1e3da55b-5ba1-4d72-8372-388176e40522/kube-auth-proxy/0.log" Jul 08 14:00:20.329173 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:20.329144 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zc2q9_7267733a-825c-47c8-83c4-8e9b5c044b00/serve-healthcheck-canary/0.log" Jul 08 14:00:20.817352 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:20.817311 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ps5d_cfb6c301-6a3c-4cc7-a065-1ec5e200a138/kube-rbac-proxy/0.log" Jul 08 14:00:20.836759 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:20.836726 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ps5d_cfb6c301-6a3c-4cc7-a065-1ec5e200a138/exporter/0.log" Jul 08 14:00:20.855769 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:20.855752 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6ps5d_cfb6c301-6a3c-4cc7-a065-1ec5e200a138/extractor/0.log" Jul 08 14:00:22.400159 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:22.400130 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tljhk/perf-node-gather-daemonset-p5npm" Jul 08 14:00:23.375562 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:23.375529 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w8rsg_028c363b-2646-4686-b821-4847944bf71d/manager/0.log" Jul 08 14:00:23.717206 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:23.717127 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-6thdj_f2769322-e620-458e-804e-0407f9efea88/manager/0.log" Jul 08 14:00:23.732984 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:23.732965 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774b77d499-6thdj_f2769322-e620-458e-804e-0407f9efea88/copy-manifests/0.log" Jul 08 14:00:25.237765 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:25.237738 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7fcdcfc8c4-t8g24_bd8bba91-ff90-4e50-b781-07c43c7ae02e/manager/0.log" Jul 08 14:00:25.266081 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:25.266012 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-72d8h_4643b9d4-ed8f-4536-b559-d28070095272/openshift-lws-operator/0.log" Jul 08 14:00:25.678035 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:25.677979 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746f76655b-glgm9_1e3da55b-5ba1-4d72-8372-388176e40522/kube-auth-proxy/0.log" Jul 08 14:00:27.194606 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:27.194584 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-746f76655b-glgm9_1e3da55b-5ba1-4d72-8372-388176e40522/kube-auth-proxy/0.log" Jul 08 14:00:31.809425 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:31.809394 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-57gt4_92436f7c-7eee-4802-b7eb-dfe846ed0420/kube-multus/0.log" Jul 08 14:00:32.219799 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.219718 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tz28k_a9385e28-6c83-44f0-aec5-a978ae6e6192/kube-multus-additional-cni-plugins/0.log" Jul 08 14:00:32.238126 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.238103 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tz28k_a9385e28-6c83-44f0-aec5-a978ae6e6192/egress-router-binary-copy/0.log" Jul 08 14:00:32.256916 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.256895 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tz28k_a9385e28-6c83-44f0-aec5-a978ae6e6192/cni-plugins/0.log" Jul 08 14:00:32.277645 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.277625 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tz28k_a9385e28-6c83-44f0-aec5-a978ae6e6192/bond-cni-plugin/0.log" Jul 08 14:00:32.300699 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.300668 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tz28k_a9385e28-6c83-44f0-aec5-a978ae6e6192/routeoverride-cni/0.log" Jul 08 14:00:32.320580 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.320559 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tz28k_a9385e28-6c83-44f0-aec5-a978ae6e6192/whereabouts-cni-bincopy/0.log" Jul 08 14:00:32.339500 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.339476 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tz28k_a9385e28-6c83-44f0-aec5-a978ae6e6192/whereabouts-cni/0.log" Jul 08 14:00:32.412625 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.412602 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-np9nc_8e451a5a-2e25-4860-810e-c7d1cd7bf260/network-metrics-daemon/0.log" Jul 08 14:00:32.431130 ip-10-0-129-157 kubenswrapper[2560]: I0708 14:00:32.431113 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-np9nc_8e451a5a-2e25-4860-810e-c7d1cd7bf260/kube-rbac-proxy/0.log"